This is an ilustration and description of the track created with A.I., I found it interesting and decided to share the results.
The tool sends an audio into LP-Music-Caps to generate a audio caption which is then translated to an illustrative image description with Llama2, and finally run through Stable Diffusion XL to generate an image from the audio.
In a dimly lit room, a lone figure sits cross-legged on a meditation mat, eyes closed in deep concentration. The steady drone of a keyboard harmony and synthesiser arrangement fills the air, creating a sense of seriousness and intensity. As the music progresses, the figure begins to twitch and tremble, as if something within them is stirring. Soft synth pads play in the lower to higher register, evoking a sense of dreaminess and otherworldliness. The mellow bells melody adds a touch of calming warmth, but the overall atmosphere remains suspenseful and spooky. As the music reaches its climax, the figure's eyes snap open, revealing a look of deep focus and determination. The low quality recording adds a sense of intimacy and secrecy to the scene, as if the listener is privy to a private moment of introspection. The ambient song's synth pad chord, smooth bass, and calming bells melody create a hypnotic atmosphere, drawing the listener deeper into the figure's inner world.
This story was created out of the picture above using llama2:
The Cosmic DJ
In the farthest reaches of the cosmos, where stars and galaxies collided in a celestial dance, there existed a man named Kael. Kael was a DJ unlike any other, for he possessed the power to weave together the very fabric of reality itself. His turntables were not mere instruments of sound, but portals to other worlds, and his beats were the rhythms of the cosmos.
One day, Kael received an invitation to perform at the most exclusive club in the universe: the Hindu Stages of Meditation. Located in the heart of a swirling nebula, this club was renowned for its transcendent atmosphere and the most talented DJs from across the cosmos. Kael knew that this was an opportunity of a lifetime, and he prepared himself for the performance of a lifetime.
As he sat in front of his keyboard, surrounded by the glow of 4K serene concept art, Kael began to play. The music was a symphony of cosmic proportions, with each note and beat weaving together in a way that defied explanation. The audience was mesmerized, their minds transported to realms beyond their wildest dreams.
But Kael's performance was not just a display of technical skill. It was a journey through the stages of meditation, a journey that he had been on for centuries. With every beat, he poured his heart and soul into the music, channeling the cosmic energy that flowed through him.
As the night wore on, Kael's performance reached new heights of intensity. The music became more complex, more intricate, and more powerful. It was as if the very fabric of reality was being torn apart and reassembled in time with the beat. The audience was entranced, their minds expanding to accommodate the cosmic rhythms.
And then, in a moment of pure transcendence, Kael's performance reached its climax. The music stopped, and in its place was a silence that was more powerful than any sound. The audience was frozen in a state of awe, their minds bathed in the cosmic light that had been unleashed.
In that moment, Kael knew that he had achieved something truly remarkable. He had brought the cosmos to life through his music, and he had transported the audience to realms beyond their wildest dreams. As the last echoes of the music faded away, Kael smiled, knowing that he had left an indelible mark on the universe.
And so, the man sitting on the floor in front of his keyboard became a legend, a myth, a cosmic DJ who had brought the universe to its knees with his music. His legacy lived on, inspiring generations of DJs to come, and his music continued to resonate through the cosmos, a testament to the power of art to transcend the boundaries of reality itself.
eugeneloza, in order to get good results, first I need a good collection of audio to train the A.I.
One thing I noticed is that you can make a voice speak any language, but the cloned voice make the same mistakes a non native speaker who doesn't master the language would do regarding pronunciation: bad pronunciation of vowels and consonants due to missing examples in the given language. That's why it would be important in my opinion to have voices in several languages.
I've never trained an A.I. to clone voices but it would be an interesting (an eventually useful) experiment.
I already tested and experimented with cloned voices and they sound quite good but I'm not sure how much training and data was needed to create them.
Ragnar Random: scripts make life easier, specially for such technology that most of the time does not even have a gui, so if you create one that works please let me know, I'm very interested.
I've never did machine learning training and just used trained so called weights to test the technology, but I think I could do that. The problem with the A.I. being trained on non open datasets I think should be discarded if you only use a voice from someone who permits his / her voice to be cloned, correct me if I'm wrong. The challenges to achieve a collection of good quality cloned voices as "ragnar Random" mentioned above would be to have a large and high quality collections of audios and another challenge would be to find people willing to let their voice be "open sourced".
There are other issues that come to my mind such as someone making a bad use of those cloned voice even if does not break the license, but nevertheless I think there is a lot of potential in a set of cloned voices for creative projects and it could eventually enhance the quality of them if well used...
Thanks Umplix! I see... one of the mixes contains several tracks from pixelsphere (cynicmusic) and 1 from Viktor Kraus. I think they both can share the CCO Public Domain license since Viktor Kraus requires atribution..
In any case I will keep them only if the authors are happy with it even if the license allow me to mix them.
Hi Commander, thanks for letting me know there was a contrast problem on that level. This is a test of how I think I can apply your idea. It looks kind of cartaoonish and it helps visualizing the edges that were otherwise hidden by the dark sides of the background.
Now I'm going to think about applying this effect by software instead of editing every foreground (and choosing which one need it), if you see the outlines of the objects on the screenshot they are done by the game so I can achieve the same effect by coding it including the shadows.
Regarding the backgrounds, I'm sorry but that level is themed on the world war and in the background there are two bombs from that war which by no means could be million light years away in an alien planet :)
This story was created out of the picture above using llama2:
eugeneloza, in order to get good results, first I need a good collection of audio to train the A.I.
One thing I noticed is that you can make a voice speak any language, but the cloned voice make the same mistakes a non native speaker who doesn't master the language would do regarding pronunciation: bad pronunciation of vowels and consonants due to missing examples in the given language. That's why it would be important in my opinion to have voices in several languages.
I've never trained an A.I. to clone voices but it would be an interesting (an eventually useful) experiment.
I already tested and experimented with cloned voices and they sound quite good but I'm not sure how much training and data was needed to create them.
Ragnar Random: scripts make life easier, specially for such technology that most of the time does not even have a gui, so if you create one that works please let me know, I'm very interested.
I've never did machine learning training and just used trained so called weights to test the technology, but I think I could do that. The problem with the A.I. being trained on non open datasets I think should be discarded if you only use a voice from someone who permits his / her voice to be cloned, correct me if I'm wrong. The challenges to achieve a collection of good quality cloned voices as "ragnar Random" mentioned above would be to have a large and high quality collections of audios and another challenge would be to find people willing to let their voice be "open sourced".
There are other issues that come to my mind such as someone making a bad use of those cloned voice even if does not break the license, but nevertheless I think there is a lot of potential in a set of cloned voices for creative projects and it could eventually enhance the quality of them if well used...
Hi, your web http://glassocean.net/ is not working.
Your tracks has been added to my project "fish fillets remake".
Your tracks can be listened during gameplay and through the internal music player.
Thank you! I'm glad you like it. It's a a pleasure to share it!
Thanks Umplix! I see... one of the mixes contains several tracks from pixelsphere (cynicmusic) and 1 from Viktor Kraus. I think they both can share the CCO Public Domain license since Viktor Kraus requires atribution..
In any case I will keep them only if the authors are happy with it even if the license allow me to mix them.
Thanks!
Sorry, my fault, it was CC0 Public Domain. I have to check all of the tracks but many of them have that license.
Maybe it is useful to provide links of the tracks I mixed and would like to share:
The hex by Pixelsphere https://opengameart.org/content/the-hex
Crystal cave by Pixelsphere: https://opengameart.org/content/crystal-cave-song18
Calm piano 1 (vaporware) https://opengameart.org/content/calm-piano-1-vaporware
Rupture by hectabex: https://opengameart.org/content/rupture
Dream ambience by tokyogeisha: https://opengameart.org/content/dream-ambience
Underwater-Ambient-Pad by isaiah658 :https://opengameart.org/content/underwater-ambient-pad
epic melodic electronic music by ISAo: https://opengameart.org/content/epic-melodic-electronic-dance-music
it does not look bad but it is hard to say with that resolution, but this is what I added, I hope it is not overdone...
Hi Commander, thanks for letting me know there was a contrast problem on that level. This is a test of how I think I can apply your idea. It looks kind of cartaoonish and it helps visualizing the edges that were otherwise hidden by the dark sides of the background.
Now I'm going to think about applying this effect by software instead of editing every foreground (and choosing which one need it), if you see the outlines of the objects on the screenshot they are done by the game so I can achieve the same effect by coding it including the shadows.
Regarding the backgrounds, I'm sorry but that level is themed on the world war and in the background there are two bombs from that war which by no means could be million light years away in an alien planet :)
Thanks!
Pages