AI Matrix, AI for drug design, SOTA deepfakes, ToonComposer, StableAvatar
Welcome to the AI Search newsletter. Here are the top highlights in AI this week.
ToonComposer is a tool that helps create animated scenes by composing 2D characters and objects in 3D environments. It uses a combination of 2D and 3D techniques to create realistic and engaging animations. This approach allows artists to focus on storytelling and creativity, rather than technical details. Read more
Matrix-Game 2.0 is an open-source, real-time, and streaming interactive world model that can generate high-quality videos on-the-fly. It uses a combination of techniques to capture complex physical dynamics and interactive behaviors, and can generate videos across diverse scenes at an ultra-fast speed of 25 FPS. This model has the potential to advance research in interactive world modeling. Read more
Yan is a foundational framework for interactive video generation, consisting of three core modules: Yan-Sim, Yan-Gen, and Yan-Edit. It enables high-quality simulation, generation, and editing of interactive video content, with applications in games, simulations, and other interactive environments. Yan's modular design allows for flexible and controllable editing, making it a powerful tool for creators. Read more
Do you prefer to watch instead of read? Check out this video covering all the highlights in AI this week
Stand-In is a powerful AI tool that can generate realistic videos from text prompts, while preserving the identity of the person in the video. It can also be used for non-human subjects, stylized videos, face swapping, and pose-guided video generation, making it a versatile tool for various applications. The technology has the potential to revolutionize video production, advertising, and social media. Read more
Matrix-3D is a new AI framework that can generate 3D worlds from a single image or text prompt, allowing for endless exploration in any direction. It uses a combination of video generation and 3D reconstruction to create high-quality, explorable 3D scenes. The framework also introduces a new dataset called Matrix-Pano, which provides a large collection of synthetic panoramic video sequences for training and testing. Read more
StableAvatar is a new AI model that can generate infinite-length, high-quality videos of avatars that are driven by audio, without the need for any post-processing. This model uses a novel approach to audio modeling and introduces a Time-step-aware Audio Adapter to prevent error accumulation and ensure smooth video generation. It outperforms existing models in both qualitative and quantitative evaluations. Read more
To celebrate reaching 500K subscribers on Youtube, we are giving a away a DJI Mini 4 Pro! This is a small, versatile, powerful drone equipped with a high resolution 48MP sensor and can capture 4K/60fps HDR video. Enter for FREE!
Voost is a new AI model that can virtually try on and try off clothes, allowing for realistic and accurate garment-body correspondence. It uses a unified and scalable diffusion transformer to learn both tasks jointly, achieving state-of-the-art results in alignment accuracy, visual fidelity, and generalization. This technology has the potential to revolutionize the fashion industry and online shopping. Read more
Researchers have developed a new robotic controller that can predict and prevent objects from slipping by adjusting the robot's hand movements. This bio-inspired approach, which mimics how humans use internal models to interact with the world, allows robots to slow down, change direction, and adapt to the position and orientation of their hands in real-time. This technology could enable robots to safely handle various physical interactions in real-world settings. Read more
FantasyPortrait is a new AI model that can create realistic and expressive animations of multiple characters from static images, using a novel approach called expression-augmented diffusion transformers. This model can handle complex emotions and multi-character scenarios, and even supports audio-driven animation and diverse character styles. It outperforms existing methods in both single- and multi-character animation scenarios. Read more
CGFlow is a new AI that can design medical drugs and provide instructions on how to make them. The tool uses a step-by-step approach to design molecules that are both biologically effective and chemically feasible to produce. This could potentially reduce the time and cost of developing new medicines. Read more
With Monica, you can use the top AI models, image generators, and video generators, all in one integrated platform. Use code AISEARCH10 to get 25% OFF 'Unlimited Annual Plan' within 24h of registration, or enjoy 10% OFF. Try it for free today!
Researchers have found that brain cells can learn faster and more efficiently than machine learning algorithms. In a study, lab-grown neural cultures integrated with silicon demonstrated dynamic plasticity and rapid adaptive reorganization in response to stimuli, outperforming advanced deep reinforcement learning algorithms. This breakthrough could lead to the development of more efficient and sustainable forms of artificial intelligence. Read more
Story2Board is a new way to generate storyboards from text, without needing to train a special model. It uses a combination of techniques to create consistent and visually diverse storyboards, while keeping the main characters looking the same throughout. This approach allows for more creative and engaging storytelling. Read more