Let's talk Lipsync

Rhubarb uses pretty much the same sinfig mechanism as Papa gallo, but it’s easier to just use it without worrying about typing because you can just go straight from wave files and forget the text baloney if you want to and it works pretty good.

Both of them work off of a statement inside the language called a switch and there is an input where you can send an integer from one to eight and there is a thing that can read from a file and change that integer from 1 to 8 in the switch statement input. Both of those programs just create a big file with nothing in it but timestamps along with numbers or letters from 1 to 8 or do. And you hook them into a switch statement.

Like I said, I’m going to open up some old laptop computers that I was doing this with a year ago, and start sharing some code with you because I love synfig and I love the community.

GitHub.com/DanielSWolf/rhubard-lip-sync version 1.13.0 seems to be the latest, In the right column under releases there’s a clickable thing where you can find it. Although when I click the link I just typed in this post it doesn’t work. Anyway if you poke around you can find it. The version of rhubarb that ships with a bunch of other animation software can also be used in a pinch just use that one

Chapter 3. Everything works great on 1.5.1, but the way switch works on Windows totally stinks and it makes it so hard that no one can stand it. So you have to use the Lineux implementation. Or else it totally sucks with Windows itis because Windows sucks all around, and send figs shouldn’t even bother supporting their b******* come on around two Linux guys. I am serious I will follow up here and show you and demonstrate just how the way dragging things into the switch when you are using the sinfig editor totally stinks on ice and windows, but is almost kind of a thing that works on Linux

Twenty characters almost

So sad. I really wanted to give some f**** about how Rhubarb is better than Papagayo.

Dude, please relax. Synfig is a small community, what did you expect? Immediate praise after uploading some random characters without even making a proper tutorial? You know my opinion on Rhubarb and other similar lip-syncing software? It’s a cheap and lazy way to make a lip syncing that will result into a soulless work. Change my mind by showing me a work that is done in Synfig and Rhubarb.

1 Like

From this

to this

:eyes: in 2 days.

2 Likes

I’m sorry I shouldn’t post on the internet while drunk. Thank you for the spirited reply though
Warning:shaky camera held in left hand. Start out at about 2 minutes and 30 seconds when I figure out how rumble studio works.

That thing I took back was too complicated, I’m going to go back to basic to explain things. I’m going to expand on the thing I create in this video, and post the code here

But I got this all figured out a year ago and never posted here, and I dropped the ball. I think this vid gets my point across, that how you make your TSV file whether with rhubarb or papagayo doesn’t really matter. it also shows how a touch interface is really nice on sinfig but it doesn’t really work quite right with the way default screen touching is, in fact under gnome under Linux you will crash synfig hard trying screen touch.

Papagayo frustrates me because it’s very picky about how you enter the text. And if you screw up the least bit it loses all of the work you put into the file and you have to start from scratch, it cannot save versions or at least it could not when I ever saw it.

Rhubarb by default just goes straight from a sound file, but it does offer language options for entering text and gaining more control that way.

1 Like

“soulless” yes absolutely. I haven’t gotten serious enough on it yet, but I’m working out ways to have like an override track on top of the soulless track so that I can personally choose a few spots I want to change but allow soulless while I take a breather. And also like I said rhubarb apparently has ways to enter text and have that take more control as is ideally what papa is doing

My crankiness about papagayo is just really that I got so frustrated that it couldn’t save my files in any way the program wanted to reuse it and it seemed really really tedious. If anything went wrong or I wasn’t up to completing the entire project in one sitting, I couldn’t get it done. Has it improved?

You can also edit the TSV file directly, and the changes take effect immediately next time you hit the play button in sinfig.

So a mistake was made, huh? It’s fine, I too allow myself some questionable things from time to time.

Yeah, the presentation isn’t on the highest level… but I got what needs to be done on Synfig part. What needs to be done on Rhubarb part though? It’s a command-line tool, most non-tech savvy people will have problems with. What parameters you should supply to it?

Wait, what code? There shouldn’t be any coding to make lip syncing… Maybe you meant command-line parameters to Rhubarb? I hope you meant that.

Wait, so Rhubarb doesn’t actually generate mouth shapes, it just provides time information? OK, so me calling it soulless was stupid, I take that back. Anyway, if you decide to make a proper tutorial that sure would be nice. Here’s how it should be structured:

  • From the very first step - generating audio file for Rhubarb, some simple sentence
  • How to feed this file to Rhubarb and get TSV file out of it
  • How to import this file in Synfig. Make a very basic character to just get the point across, what result we should get (a character speaking). You do not need to put one of your cool characters that you care about as an example