My main use case is modifying youtube videos of tech tutorials where the speaker overlays a video of themselves in a corner of the video. drawvg is used to blackout that area of the video. I'm sure some viewers like having a visible talking head shown on the same screen as the code but I find the constant motion of someone's lips moving and eyes blinking in my peripheral vision extremely distracting. Our vision is very tuned into paying attention to faces so the brain constantly fighting that urge so it can concentrate on the code. (A low-tech solution is to just put a yellow sticky know on the monitor to cover up the speaker but that means you can't easily resize/move the window playing the video ... so ffmpeg to the rescue.)
If the overlay was a rectangle, you can use the older drawbox filter and don't need drawvg. However, some content creaters use circles and that's where drawvg works better. Instead of creating a separate .vgs file, I just use the inline syntax like this:
ffmpeg -i input.webm -filter_complex "[0:v]drawvg='circle 3388 1670 400 setcolor black fill'[v2];[0:a]atempo=1.5[a2]" -map "[v2]" -map "[a2]" output.mp4
(The author of the drawvg code in the git log appears to be the same as the author of this thread's article.)
[1] https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/016d767c8e9d...
I got various bits and pieces together and experimented with doing things like driving Big Trak gearboxes (remember? J Bull Electrical used to advertise them in every magazine, along with all sorts of fascinating old shite) with an interface plugged into my ZX Spectrum, but I never actually built one.
Funnily enough I was thinking about that the other day, and how sad it is that schools like my son's primary school just have very locked-down iPads for the children to use instead of the BBC Micros we grew up with (I'm guessing you're more approximately my age than primary school age, and those things were in schools well into the early 2000s. Bombproof.) that could be endlessly tinkered with.
Anyway the guy next door does a lot of 3D printing and it's never been easier to draw PCBs and get them made or even etch them at home (it's the drilling bit I hate). So maybe now EBv2.0 is five, it's time to dig out that issue of HE and start transcribing stuff into Kicad and Blender :-)
Still, I find the syntax it uses horrible:
ffmpeg -an -ss 12 -t 3 -i bigbuckbunny.mov -vf 'crop=iw-1, drawvg=file=progress.vgs, format=yuv420p' -c:v libvpx-vp9 output.webm
My poor, feeble brain, though, has a hard time deducing all of this. Yes, I can kind of know what it does to some extent ... start at 12 seconds right? during 3 seconds ... apply the specified filter in the specified format, use libvpx-vp9 as the video codec ... but the above example is somewhat simple. There are total monsters in actual use when it comes to the filter subsystem in ffmpeg. Avisynth was fairly easy on my brain; ffmpeg does not, and nobody among the ffmpeg dev team seems to think that complicated uses are an issue. I even wrote a small ruby script that expands shortcut options as above, into the corresponding long names, simply because the long names are a bit easier to remember. Even that fails when it comes to complex filters used.
It's a shame because ffmpeg is otherwise really great.
For what it's worth, LLMs are a great tool for both composing and understanding ffmpeg commands.
And if you want something more verbose / easier to read you can use something like https://github.com/kkroening/ffmpeg-python (with LLMs) as well
mpv doesn't run on iPad so it's better for my situation to just burn the blackout into a new video. I actually do a lot more stuff than drawvg (also rescale, pts, framerate,etc) in filter_complex but left the rest of it out for the HN comment so the example is more readable.
I suppose it might be possible to use mpv with a custom shader mask glsl code to blackout circular areas of the screen.
>you sure can crop the video
Cropping the video is straightforward with no information loss when the geometry of presentation and the speaker is laid out like these: https://www.youtube.com/@MeetingCPP/videos
But cropping the following video by shrinking the boundaries of the rectangle until the circle overlay is not visible would result in too much of the text being cut off: https://www.youtube.com/watch?v=nUxuCoqJzlA
Scrub that video timeline to see the information that would be chopped off. For that, it's better to cover up only the circle overlay with a blacked out disc.
[1] https://developer.mozilla.org/en-US/docs/Web/SVG/Reference/E...
It's a shame SVG doesn't. Many shapes can be specified much more concisely.
It may seem "dead" but ultimately it just helps you build CLI commands in a more sane way, the CLI interface to ffmpeg has been consistent for a long time. Only thing that may change is individual filters which you can just give raw to ffmpeg-python.
I remember when I was heavily using it last year I found a fork that seemingly had more sane typing or something but since LLMs last year didn't know about the newer lib but could write decent ffmpeg-python code I stuck with it and it did the job.
PlaygroundLanguage ReferenceSource Code
drawvg is a FFmpeg filter, available since 8.1, to render vector graphics on top of video frames.
The render is done by executing a script written in its own language, called VGS (Vector Graphics Script). The script consists of a series of commands to describe 2D graphics, which are rasterized using the Cairo library.
VGS is not intended to be used as a general-purpose language. Since its scope is limited, it prioritizes being concise and easy to use. The syntax is heavily inspired by languages like Magick Vector Graphics, or SVG's <path>. Some features of the syntax (like using whitespaces to separate arguments) are also present in languages like TCL or shell scripts. Many command names are taken from PostScript. VGS is fully documented in the language reference.
Scripts can use FFmpeg expressions to describe graphics dynamically, so they can compute coordinates based on frame dimensions, frame metadata, generate random values, read pixel colors, etc.
This is a short list of examples to showcase how to integrate the drawvg filter with other filters in FFmpeg.
The Playground has a gallery with more examples, focused on the capabilities of the VGS language.
The variable t can be used to compute one of the angles of the arcn command. Then, we can create an animation like this:
The script can be rendered directly on top of a video:
Output
Shell
ffmpeg \
-an \
-ss 12 -t 3 -i bigbuckbunny.mov \
-vf 'crop=iw-1, drawvg=file=progress.vgs, format=yuv420p' \
-c:v libvpx-vp9 \
output.webm
This example uses clips from the Big Buck Bunny movie, available under CC BY 3.0 license.
The cropdetect filter calculates the necessary cropping parameters to remove black borders around a video. These parameters are added to each frame as metadata.
drawvg can access the output of cropdetect with the getmetadata command. The following example draws a red rectangle to represent the calculated area by cropdetect.
Output
Shell
ffmpeg \
-an \
-i highway.mp4 \
-vf 'cropdetect, drawvg=file=cropdetect.vgs, format=yuv420p' \
-c:v libvpx-vp9 \
output.webm
This example uses the video Night Drive on Highway with Passing Cars, free to use by the pexels license.
This example creates a transition similar to the circlecrop transition of the xfade filter, but the circle can be positioned anywhere, not only at the center of the frame.
Output
circlecrop.vgs
rect 0 0 w h
setvar d (sin(PI / 2 * (t - ts)))
circle (4 * w / 5) (3 * h / 5) (max(w, h) * (1 - d))
eofill
circlecrop.filter
crop = iw-1 ,
drawvg = file=circlecrop.vgs : enable='gt(t,0.8)' ,
format = yuv420p
Shell
ffmpeg \
-an \
-ss 14 -t 4.2 -i bigbuckbunny.mov \
-/vf circlecrop.filter \
-c:v libvpx-vp9 \
output.webm
This example uses clips from the Big Buck Bunny movie, available under CC BY 3.0 license.
Another way to create custom transitions is to use the alphamerge and overlay filters, with a mask rendered with a drawvg script.
This is the output of the drawvg script:
alphamerge can set these frames as the alpha channel of a video. Then, use overlay to put the video with the mask on top of another one.
Output
transition.filter
[0] split [mask-bg] [mask-delay] ;
[mask-bg] drawvg = file=transition.vgs [bars] ;
[mask-delay] [bars] concat [mask] ;
[1] [mask] alphamerge [a] ;
[2] [a]
overlay = enable='lt(t,4)' ,
crop = iw-1 ,
format = yuv420p
Shell
ffmpeg \
-f lavfi -i 'color=white:s=853x480:r=24:d=2' \
-ss 16 -t 4 -i bigbuckbunny.mov \
-ss 7:51 -t 6 -i bigbuckbunny.mov \
-/filter_complex transition.filter \
-an \
-c:v libvpx-vp9 \
output.webm
This example uses clips from the Big Buck Bunny movie, available under CC BY 3.0 license.
The function p(x, y) returns the color of a pixel at the given coordinates. It can be used to apply pixelization to a frame, similar to the pixelize filter.
Instead of rectangles, the shape used for pixelization are rhombuses, and each one has a thin border to highlight its outline.
The output below shows the original frame on the left, and the frame updated by the drawvg script on the right:
Output
pixelate.filter
[0] scale = iw/1.5 : -2 , split [a] [b] ;
[a]
pixelize ,
drawvg = file=pixelate.vgs ,
pad = iw*2+30 : ih+20 : iw+20 : ih+10
[a0] ;
[a0] [b]
overlay = 10 : 10 ,
format = yuv420p
Shell
ffmpeg \
-an \
-ss 1 -t 18 -i bigbuckbunny.mov \
-/filter_complex pixelate.filter \
-c:v libvpx-vp9 \
output.webm
This example uses clips from the Big Buck Bunny movie, available under CC BY 3.0 license.
drawvg can be combined with the displace filter to create a wave effect:
First, a drawvg script renders horizontal rectangles with different shades of gray. Then, boxblur is used to soften the transition between rectangles. This image is used as the xmap input for displace. The output below contains the intermediate images.
Output
waves.filter
[0] crop = iw-1, split [source0] [source1] ;
[source0]
drawvg = rect 0 0 w h setcolor gray fill ,
loop = -1 : 1 : 0 ,
split
[gray0] [gray1] ;
[gray0]
drawvg = file=waves.vgs ,
split
[xmap_src0] [xmap_src1] ;
[xmap_src0]
boxblur = h/40 ,
split
[xmap0] [xmap1] ;
[source1] [xmap1] [gray1]
displace = mirror ,
pad = iw*2+30 : ih*2+30 : 10:10
[pad0] ;
[pad0] [xmap_src1] overlay = 10 : h+20 [pad1] ;
[pad1] [source1] overlay = w+20 : 10 : shortest=1 [pad2] ;
[pad2] [xmap0]
overlay = w+20 : h+20 : shortest=1 ,
drawtext = text='drawvg' : y=h-20 : x=20 : y_align=baseline : fontsize=32 : fontcolor=white ,
drawtext = text='boxblur' : y=h-20 : x=w/2+20 : y_align=baseline : fontsize=32 : fontcolor=white ,
drawtext = text='Source' : y=h/2-20 : x=w/2+20 : y_align=baseline : fontsize=32 : fontcolor=white ,
format = yuv420p
Shell
ffmpeg \
-an \
-ss 5 -t 10 -i bigbuckbunny.mov \
-/filter_complex waves.filter \
-c:v libvpx-vp9 \
output.webm
This example uses clips from the Big Buck Bunny movie, available under CC BY 3.0 license.
Is there something similar that supports shaders? Like metal / wgsl / glsl or something?
Sounds like a fun project...
On the other hand, when I compared the binaries (ffmpeg, ffprobe, ffplay) I downloaded the other day with the ones I had installed since around September, they where almost 100 MB larger. I don't remember the exact size of the old ones but the new ones are 640 MB, the old ones well under 600 MB. The only difference in included libraries was Cairo and the JPEG-XS lib. So while I think a bunch of new ML models would be really cool, maybe they don't want to go down that route. But some kind of pluggable system with accelerated ML models would be helpful I think.