Need your help creating a masking tool!

Discussion in 'General Discussion' started by Centurione, Apr 14, 2017.

  1. Centurione

    Centurione New Member

    Hi guys! It's been a while, hope you've got some masterpieces in the making ;)

    I kind of don't feel like making AMVs anymore however my brain still feels bad after doing all that masking for my projects so I figured out I could try making a nice tool for that as my Engineering Thesis.

    I have signed up to develop a tool which would do masking automatically after you've given it a sample mask in the scene or marked an area which you would like to mask. Pretty much like rotobrush in After Effects but I have an ambition to actually make it work well in most cases, including animations :D

    In order for these algorithms to have any chance to work I need a hella big dataset of masked scenes and, as seems logical, I am incapable of producing it on my own.

    So here goes my request: if you happened to do some masking for your video and wished to help me out, I would be most grateful if you provided me with your masked scene and your name and surname so that I could mention you in my thesis. The data would be used for that purpose exclusively and wouldn't be shared with absolutely anyone.

    If you wished to help me out with my project here are some options on how you can do it:

    1) Most conviniently would be providing me with the scene before masking and a scene after masking in a format in which the masked area would be entirely white and the background entirely black.

    Here is an example:
    Preferred format is providing me with 2 files or folders:
    1) With original scene in lossless AVI or PNG/JPG sequence
    2) Masked scene in lossless AVI or PNG/JPG sequence

    The above option however requires some work from you so I basically accept any format which would make it possible for me to get the result mentioned above on my own in a faster way than masking it out myself, which includes:

    2. A project (Vegas or After Effects) with masking done and the source scene.
    3. A rendered result mentioned in 1) but with greenscreen/bluescreen/whatever makes it possibly for me to key the color out and information about the source (series, episodes, things like that).

    Or, well, whatever makes it possible for me to analyse the source material and create a black-and-white result.

    The source doesn't really matter: whether it's animation, live action footage, whatever masked.

    Email address: Please provide me with any possible way to download what you want to share with me.

    I know this is not the tiniest thing to ask and I am in my very initial phase of planning my work but since I know it might be hard to get the amount of data needed, I have to start early.

    My thesis should be finished by January 2018. If the machine learning and automatic masking doesn't work out how I want it to, I'm going to at least develop the most convinient masking tool there is ;)

    Feel free to ask any questions or doubts. Or maybe you know about a similar tool that already exists? I'm only aware of rotobrush and I've been out of hobby for a couple of years ^^'

    Thank you for taking your time to read this and I well, if you wish to help me out, I will be moooooost grateful.

    Whoa, what an essay!

    Hi! October is here!
    I'm still collecting and generating data myself since it's the most important part of the project. I need to have my thesis written by mid December. By then I want to have a model with noticeable results and a web interface so that you could make it running like this.
    After that I plan on developing the tool as my own project to make it usable for end-users (you <3) and, of course, imporve the model if I manage to get ANY reasonable result during my work for thesis.
    So basically, keep them coming!
    Last edited: Oct 12, 2017
    Axel, Iren S.S., FoxJones and 5 others like this.
  2. Fall_Child42

    Fall_Child42 Moderator

    Here is something that Kind of does it, you might be able to reverse engineer it and make it multi frame.

    It's for creating automatic Depth maps of Scenes for use in stereoscopic stuff.
  3. leolide

    leolide New Member

    does transparent png work too?
  4. Centurione

    Centurione New Member

    Sure it does!
  5. MapleMAD

    MapleMAD Damsel in Defense

    Hi Centurione, I am a big fan of your works!

    I would like to recommend a free after effect's plugin developed by a chinese who goes by ID 人间之表, which I think is a pretty convinient masking tools available for free, so maybe you can have a look at the plugins and who knows it might help with your project ^^

    Download link:

    Anyways it's not automated like rotobrush and it will be messy if the background color is very similar to the things that you want to key out.
    A few mask path on the unwanted backgrounds and voila.

    Quick test on the plugins:

    Edit: Quick guide on this, create Mask on unwanted spots, then put the mask into Flood Seed.
    There are a few features which will improve the final result like feather and clean up edge. I'm more of a pen tools person, so I didn't know much details about how to effectively using this though.

    As for masked scenes I will come back to you when I finished render them out in black background ^^
    It's great to see someone working on an automated masking tools, wish you all the luck !
    Last edited: Apr 14, 2017
  6. Centurione

    Centurione New Member

    I have to say this tool looks awesome! Thank you for showing it to me (and I suppose many AMV makers as well), definitely gonna look into it!
    And thank you if you plan on sending some samples <3
    MapleMAD likes this.
  7. Lirinis

    Lirinis New Member

    That's a really fascinating project. A working roto tool for animation would be adorable.

    Did you try to search youtube for [anime mask]? There's a lot of anime greenscreen material up there. Of varied quality. The only problem is that those masks are usually provided without the source scene. But maybe you could ask the uploaders and maybe some of them still have the projects.

    I'm sure there are several video datasets premade specifically for background subtraction research. Just google them.

    I hope you'll be sharing your progress with us. I'd really like to follow it.
  8. Centurione

    Centurione New Member

    Sure I will be sharing my progress! So far I'm only starting to collect data for it.

    There are toold for background subtraction but I wanted to make it particularly for video masking so that it doesn't analyse only the current frame but all of them at once to catch some kind of context.
    Got some youtube links already, I can recognize most of the shows so I may actually be able to find the source scenes for most of these :)
    Thanks for your support!
  9. ZoroIttoryu

    ZoroIttoryu Pirate Hunter

    This seems like the best tool someone could use for amvs nowdays, most edits use masking some way or another.
    Would be extremely usefull for more than many editors! Good luck with it, i will also try to give you masks when i actually mask something :p
  10. Replay

    Replay Administrator Staff Member

    Good luck! I be happy to beta test the plugin when you reach that far ^^
  11. Cmoididi

    Cmoididi WeedLord

    you want some cmoididi roto haha ? if i have some time soon i'll send some 'sans titre' green screen (keying/masked) and some random anime roto, seem to be a cool project.
  12. Voice From Moon

    Voice From Moon Member New

    I hate masking in general so this seems very nice project to look up to.
    In what naming pattern everything preferably should be sent out? Folder structure? file format(codecs, settings. etc)?
    While it most likely won't do any major difference to me(or anyone else), I suppose It could save some time for you to receive required files in standardized way, and not reshuffling them for yourself every single time.
    FoxJones likes this.
  13. Centurione

    Centurione New Member

    Thanks for your support!

    Sure! I will be glad to have testers, expecially for the interface.

    Cmoididi roto would be da best : 3 Thanks!

    I will have to restructure everything pretty strongly anyway and will probably name files by following numbers so don't bother about that :) About format - I'd prefer it uncompressed but it doesn't really matter either for now since I have to talk about the exact way of processing files with my promotor and then I will decide which format I would use.
  14. Iren S.S.

    Iren S.S. Guns N' Vincents

    That's a handy tool you're working on. I will try to share those miserable couple of masks I did throughout these years once my exams are over ^^"
  15. ailynerie

    ailynerie "El Axel"

    Sent in an email! :LuciBro:
    Centurione likes this.
  16. Centurione

    Centurione New Member

    You are the first contributor! These are perfect! Thank you so much!
  17. Centurione

    Centurione New Member

    Just a reminder! The project will be ongoing until Janurary 2018 so I will be glad to receive some uploads until even December 2017 :)
  18. Konnor-Kun

    Konnor-Kun Konnor Daisako

    I'm Happy to help in you proyect
  19. Centurione

    Centurione New Member

    Hey guys! Just a reminder! I'm still in a HUGE need of masks so drop me some if you wish so. Best way to contact me is on (lost access to some social media accounts :( )
  20. Iren S.S.

    Iren S.S. Guns N' Vincents

    Sent in a thingy too ^^

Share This Page