Ghulam Mujtaba

With the surge in emerging technologies such as Metaverse, spatial computing, and generative AI, the application of facial style transfer has gained much interest from researchers and startups enthusiasts alike. StyleGAN methods have paved the way for transfer-learning strategies that could reduce the dependency on the vast data available for the training process. However, StyleGAN methods tend to need to be more balanced, resulting in the introduction of artifacts in the facial images. Studies such as DualStyleGAN proposed multipath networks but required the networks to be trained for a specific style rather than simultaneously generating a fusion of facial styles. In this paper, we propose a Fusion of STyles (FIST) network for facial images that leverages pretrained multipath style transfer networks to eliminate the problem associated with the lack of enormous data volume in the training phase and the fusion of multiple styles at the output. We leverage pretrained styleGAN networks with an external style pass that uses a residual modulation block instead of a transform coding block. The method also preserves facial structure, identity, and details via the gated mapping unit introduced in this study. The aforementioned components enable us to train the network with minimal data while generating high-quality stylized images, opening up new possibilities for facial style transfer in emerging technologies. Our training process adapts curriculum learning strategy to perform efficient, flexible style, and model fusion in the generative space. We perform extensive experiments to show the superiority of the proposed FISTNet compared to existing state-of-the-art methods.

Systems and methods are provided herein for generating a summary for a piece of content using a thumbnail container. This may be accomplished by a system receiving a thumbnail container related to a piece of content. The system may also receive user information, a device characteristic, and/or content information related to the piece of content and use the received data to select a machine learning model. The selected machine learning model can identify one or more thumbnails of the thumbnail container as a thumbnail of interest to a user. The system can then generate a summary of the piece of content based on the thumbnail identified by the machine learning model and display the generated summary for the user.

Generating video highlights in the form of animated graphics interchange formats (GIFs) has significantly simplified the process of video browsing. Animated GIFs have paved the way for applications concerning streaming platforms and emerging technologies. Existing studies have led to large computational complexity without considering user personalization. This paper proposes lightweight method to attract users and increase views of videos through personalized artistic media, i.e., static thumbnails and animated GIF generation. The proposed method analyzes lightweight thumbnail containers (LTC) using the computational resources of the client device to recognize personalized events from feature-length sports videos. Next, the thumbnails are then ranked through the frame rank pooling method for their selection. Subsequently, the proposed method processes small video segments rather than considering the whole video for generating artistic media. This makes our approach more computationally efficient compared to existing methods that use the entire video data; thus, the proposed method complies with sustainable development goals. Furthermore, the proposed method retrieves and uses thumbnail containers and video segments, which reduces the required transmission bandwidth as well as the amount of locally stored data. Experiments reveal that the computational complexity of our method is 3.73 times lower than that of the state-of-the-art method.

This paper proposes a novel lightweight thumbnail container-based summarization (LTC-SUM) framework for full feature-length videos. This framework generates a personalized keyshot summary for concurrent users by using the computational resource of the end-user device. State-of-the-art methods that acquire and process entire video data to generate video summaries are highly computationally intensive. In this regard, the proposed LTC-SUM method uses lightweight thumbnails to handle the complex process of detecting events. This significantly reduces computational complexity and improves communication and storage efficiency by resolving computational and privacy bottlenecks in resource-constrained end-user devices. These improvements were achieved by designing a lightweight 2D CNN model to extract features from thumbnails, which helped select and retrieve only a handful of specific segments. Extensive quantitative experiments on a set of full 18 feature-length videos (approximately 32.9 h in duration) showed that the proposed method is significantly computationally efficient than state-of-the-art methods on the same end-user device configurations. Joint qualitative assessments of the results of 56 participants showed that participants gave higher ratings to the summaries generated using the proposed method. To the best of our knowledge, this is the first attempt in designing a fully client-driven personalized keyshot video summarization framework using thumbnail containers for feature-length videos.

This paper proposes a lightweight methodology to attract users and increase views of videos through personalized artistic media i.e., static thumbnails and animated Graphics Interchange Format (GIF) images. The proposed method analyzes lightweight thumbnail containers (LTC) using computational resources of the client device to recognize personalized events from feature-length sports videos. In addition, instead of processing the entire video, small video segments are used in order to generate artistic media. This makes our approach more computationally efficient compared to existing methods that use the entire video data. Further, the proposed method retrieves and uses thumbnail containers and video segments, which reduces the required transmission bandwidth as well as the amount of locally stored data that are used during artistic media generation. After conducting experiments on the NVIDIA Jetson TX2, the computational complexity of our method was 3:78 times lower than that of the state-of-the-art method. To the best of our knowledge, this is the first technique that uses LTC to generate artistic media while providing lightweight and high-performance services on resource-constrained devices.

Click-through rate (CTR) is a critical metric to boost the popularity of newly published videos on streaming platforms. Humans and human-like characters play a significant role in GIF selection and improving the CTR of the video. This paper proposes a new lightweight method to generate human character-oriented animated GIFs using the end-user device’s computational capabilities. Instead of analyzing full video, the proposed method analyzes the lightweight thumbnail containers to decrease computational complexity in the GIF generation process. Moreover, it uses the segment to generate the GIF and reduced valuable network bandwidth and storage demands in the user end. A feed-forward 2D deep neural network trained on the CelebA dataset is designed to detect humans or humanlike characters and their gender. Experimental evaluations and results performed in 10 full videos showed that the proposed method is 2.34 times more computationally efficient than the SoA approach. The proposed method is designed to support end-user devices with different computational capabilities.

This paper proposes a novel, lightweight method to generate animated graphical interchange format images (GIFs) using the computational resources of a client device. The method analyzes an acoustic feature from the climax section of an audio file to estimate the timestamp corresponding to the maximum pitch. Further, it processes a small video segment to generate the GIF instead of processing the entire video. This makes the proposed method computationally efficient, unlike baseline approaches that use entire videos to create GIFs. The proposed method retrieves and uses the audio file and video segment so that communication and storage efficiencies are improved in the GIF generation process. Experiments on a set of 16 videos show that the proposed approach is 3.76 times more computationally efficient than a baseline method on an Nvidia Jetson TX2. Additionally, in a qualitative evaluation, the GIFs generated using the proposed method received higher overall ratings compared to those generated by the baseline method. To the best of our knowledge, this is the first technique that uses an acoustic feature in the GIF generation process.

Movie trailers are prepared using a one-size-fits-all framework. These days, however, streaming platforms seek to overcome this problem and provide personalized trailers via the investigation of centralized server-side solutions. This can be achieved by analyzing personal user data, and can lead to two major issues: privacy violation and enormous demand in computational resources. This paper proposes an innovative, low-power, client-driven method to facilitate the personalized trailer generation process. It tackles the complex process of detecting personalized actions in real-time from lightweight thumbnail containers. The HTTP live streaming (HLS) server and client are locally configured to validate the proposed method. The system is designed to support a wide range of client hardware with different computational capabilities and has the flexibility to adapt to network conditions. To test the effectiveness of this method, twenty-five broadcast movies, specifically in the western and sports genres, are evaluated. To the best of our knowledge, this is the first-ever client-driven framework that uses thumbnail containers as input to facilitate the trailer generation process.

Mobile devices have been increased exceptionally in recent years, consequently data generation has also been raised exceptionally. Most of the data generated by mobile devices is transferred to servers for processing and storage. Managing security of mobile data is a necessary feature of every network and mostly encryption is used to avoid security breaches. The major challenge is that, mobile devices are very small with shortage of resources, on the other hand encryption of data requires extra energy. It is necessary to minimize energy requirements for encryption of data. For this experimental research, an android based application is developed, which optimize energy requirements for both single and double encryption techniques. AES and Blowfish encryption algorithms are used with different files sizes to test the energy requirements for single encryption, it is also examined that energy consumed by Blowfish is 119.311% more than AES. For double encryption methods, AES–Blowfish, Blowfish–AES and XTS–AES combinations of algorithms are used and energy usage is gathered. In double encryption XTS–AES consumed 13.26% less power consumption as compared to AES–Blowfish and 44.97% less then Blowfish–AES combination methods. Results of experiments revealed that AES is more energy efficient for single encryption and for double encryption XTS AES combination requires less energy.