2023 updates

I've made some changes to the blog, I'll try to sum­ma­rize it here for the grand to­tal ze­ro num­ber read­ers this blog has. This is go­ing to be tech­ni­cal, and most­ly a rant, so feel free to ig­nore this.

It all start­ed with me try­ing to write a new ar­ti­cle, when I want­ed to put time­codes in­to the text. For op­ti­mal us­age, it should be click­able, and the video should just jump to it, right? Un­for­tu­nate­ly I don't think that can be done with­out javascript (es­pe­cial­ly if you have two video, and you want to jump to the video that was viewed last by the user...), so I quick­ly re­al­ized I'll need to bite the bul­let and add some javashit to the page. Don't wor­ry, every­thing should work with dis­abled JS (ex­cept for one small thing with the videos, more on that lat­er).

So I've been look­ing around at var­i­ous open source JS video play­ers, in the end I've cho­sen plyr, and I've start­ed to play around with it... and I re­al­ized that it's usu­al web-de­vel­op­er qual­i­ty shit (i.e. ut­ter garbage). And a bloat­ed mess. I've looked at a few oth­er al­ter­na­tives, they're ei­ther pro­pri­etary, have hor­ri­ble li­cens­es, or doesn't know the fea­tures I'd need. So I down­loaded the plyr re­po, and re­moved most of the bloat and spy­ware (sor­ry, I can't cat­e­go­rize YouTube and ad­verts in­te­gra­tion as any­thing but spy­ware), fixed some bugs (I need­ed to add a we­b­com­po­nent poly­fill, so it works in Pale Moon, I have no idea how this shit works in IE11 when the rec­om­mend­ed poly­fill doesn't in­clude we­b­com­po­nents poly­fill)... but at this point I was al­ready run­ning down the rab­bit hole. It's skin looks hor­ri­ble (typ­i­cal mod­ern garbage that tries to look like An­droid as much as pos­si­ble), it can be cus­tomized through some CSS vari­ables, but you can bare­ly use it for any­thing be­sides re­col­or. Like, I want­ed to get rid of that huge and point­less padding around the con­trols, it's not fuck­ing chrome, It looks like it can be set by --plyr-control-spacing... but if I set that, it al­so sets all padding be­tween the con­trols, so if I set it to ze­ro, con­trols will be squished on the play­er. Of course, one small CSS over­ride can fix this, but to fix every prob­lem, I'd like­ly need to write so many over­rides (or ed­it the source style) that it would amount to a rewrite. Sigh.

Gallery#

So any­way, while I was vent­ing about how shit this play­er is, I re­mem­bered that I al­ways want­ed a gallery that's a tiny lit­tle bit more us­able than just open­ing im­ages in new tabs. Orig­i­nal­ly I didn't have a gallery, be­cause do­ing it in pure CSS is next to im­pos­si­ble (es­pe­cial­ly if you want it to have fea­tures), but now that I have JS...

And this is where things went wrong. Of course, I could have used some pre­made gallery, in­ject 15 gi­ga­bytes of left­pad like de­pen­den­cies, that of course fails spec­tac­u­lar­ly in any­thing but the lat­est chrome/fire­fox/sa­fari... but in­stead, I've de­cid­ed to write my own. I've had to write some ba­sic JS to get that play­er work­ing any­way. So down the rab­bit hole again. I don't want to write too much about this, this was prob­a­bly the most fun part of this whole. Yes, javashit is a re­tard­ed lan­guage, but un­for­tu­nate­ly I've come to the con­clu­sion that all the ecosys­tem that was built around it is even more re­tard­ed. We've reached (not ex­act­ly il­lus­tri­ous) achieve­ment where javashit is not the worst as­pect of web de­vel­op­ment. So just click on the im­ages and en­joy the full-page gallery with some unique fea­tures, like in­te­ger scal­ing, pro­vid­ed you have JS en­abled and your devicePixelRatio is 1.

HiDPI garbage#

It prob­a­bly doesn't work. See, some time ago, when they start­ed to cre­ate dis­plays with high­er den­si­ties, they re­al­ized that web­sites that used px when they should have used em and friends will be un­read­ably small in those shiny new screens. The so­lu­tion? Just change the de­f­i­n­i­tion of the pix­el. In CSS, px no longer means one pix­el. It's an ab­stract unit that can mean any­thing from 1e-308 pix­els to 1e308 pix­els. And usu­al­ly it's not an in­te­ger. So, what can you use to get some­thing that's 1 pix­el? Noth­ing. I mean, look at this stack­over­flow ques­tion. Javashit codes that fill the screen just to cre­ate some­thing that's 100x100 pix­els large. Re­al­ly? Are you fuck­ing nuts? This is the fuck­ing state of the fuck­ing web in 2023, you can't make a 100x100 pix­els large shit. They just say use SVG. Yeah, please give me a time ma­chine, so I can tell peo­ple in 1989 to use SVG when you have a few MHz CPU with less than 1 MiB of RAM. And you won­der while every mod­ern crap looks the same, this whole fuck­ing tech­nol­o­gy is in­ca­pable of cre­at­ing any­thing more com­pli­cat­ed than sin­gle col­ored rec­tan­gles where you have no con­trol over the size of said rec­tan­gles.

I have some hack in the gallery code to com­pen­sate for non-1 devicePixelRatio val­ues, but I'm pret­ty sure it will break with some browsers. And do­ing this in pure CSS is damn im­pos­si­ble, you have dppx, which is sup­posed to rep­re­sent how many (re­al) pix­els you have in one (fucked-up CSS) pix­el, but you can on­ly use it in (range) me­dia queries. I was think­ing about gen­er­at­ing a file, like

@media (max-resolution: 1.000000dppx) :root { --dppx: 1.000000 }
@media (max-resolution: 1.000001dppx) :root { --dppx: 1.000001 }
@media (max-resolution: 1.000002dppx) :root { --dppx: 1.000002 }
@media (max-resolution: 1.000003dppx) :root { --dppx: 1.000003 }
@media (max-resolution: 1.000004dppx) :root { --dppx: 1.000004 }
@media (max-resolution: 1.000005dppx) :root { --dppx: 1.000005 }
@media (max-resolution: 1.000006dppx) :root { --dppx: 1.000006 }
@media (max-resolution: 1.000007dppx) :root { --dppx: 1.000007 }
@media (max-resolution: 1.000008dppx) :root { --dppx: 1.000008 }
@media (max-resolution: 1.000009dppx) :root { --dppx: 1.000009 }
@media (max-resolution: 1.000010dppx) :root { --dppx: 1.000010 }
@media (max-resolution: 1.000011dppx) :root { --dppx: 1.000011 }
@media (max-resolution: 1.000012dppx) :root { --dppx: 1.000012 }
@media (max-resolution: 1.000013dppx) :root { --dppx: 1.000013 }
@media (max-resolution: 1.000014dppx) :root { --dppx: 1.000014 }
@media (max-resolution: 1.000015dppx) :root { --dppx: 1.000015 }
@media (max-resolution: 1.000016dppx) :root { --dppx: 1.000016 }
@media (max-resolution: 1.000017dppx) :root { --dppx: 1.000017 }
@media (max-resolution: 1.000018dppx) :root { --dppx: 1.000018 }
@media (max-resolution: 1.000019dppx) :root { --dppx: 1.000019 }

do it from 0 to 10, and you have a 640 MiB CSS file (but on­ly 48 MiB af­ter gzip com­pres­sion!). But that's huge and the pre­ci­sion is lim­it­ed. And would prob­a­bly DoS every brows­er. An­oth­er idea was to cre­ate some­thing like this:

@import "x.css?l=0&h=5" max-resolution: 5dppx;
@import "x.css?l=5=h=10" not max-resolution: 5dppx;

and have a serv­er-side com­po­nent that would gen­er­ate these x.css files to have a kind of bi­na­ry search for the val­ue of res­o­lu­tion. It should be more ac­cu­rate than the big pre­gen­er­at­ed list above, con­sume way less band­width, but it would in­crease la­ten­cy a lot, and it would al­so no longer be a sta­t­ic site.

So TL;DR if you use a HiDPI screen, you're fucked.

Image formats#

Up un­til now I gen­er­al­ly used WebP, ex­cept when PNG or JPG was bet­ter. In the mean­time, JXL gained some trac­tion, I al­so looked in­to HTML's <picture> el­e­ment, so in the end I got rid of jew­gle's shilled im­age for­mat. The more I look at it, the more I re­al­ize how ut­ter­ly crap it is. First, it on­ly sup­ports YUV420 in the lossy mode (even though VP9 sup­port YUV444), which means it's com­plete­ly un­suit­able for any­thing that's not a high res­o­lu­tion pho­to­graph. I've tried to use it for some thumb­nails, but 99% WebP pro­duced worse re­sults than 70% JPG with twice the file size. So ba­si­cal­ly you can on­ly use the loss­less com­pres­sion. There's a "near loss­less" mode, which us­es the loss­less mode of WebP, but pre­process­es the im­age, so it com­press­es bet­ter. That's like the on­ly sane way to use WebP if you don't en­code a pho­to­graph, but since it still us­es the loss­less mode, the files en­cod­ed this way are still pret­ty big. And us­ing plain pngquant pro­duces bet­ter re­sults many times any­way.

And the sec­ond prob­lem is, the lack of any in­ter­lac­ing/pro­gres­sive im­age sup­port. Jew­gle says it is to re­duce CPU us­age, but that whole an­swer sounds like some di­ver­si­ty hire bull­shit. You don't have to re­fresh the screen with each byte re­ceived you fuck­ing dimwit. In­cre­men­tal de­cod­ing is not an al­ter­na­tive, it's a fuck­ing stan­dard fea­ture in fuck­ing every im­age de­coder in used fuck­ing every brows­er, and it means if you have a half down­loaded im­age, you'll see the up­per half of it. Not a low res­o­lu­tion one that you have with in­ter­laced im­ages. You man­aged to kill a fea­ture that JPG and PNG sup­port­ed for more than 30 years. You man­aged to de­stroy user ex­pe­ri­ence on slow­er in­ter­net con­nec­tions. And this is a for­mat that was sup­pos­ed­ly cre­at­ed for the web. Is there any peo­ple left with pos­i­tive IQ score at jew­gle!? Sigh.

So, any­way, I ex­ter­mi­nat­ed all WebP im­ages from this site, every­thing should be avail­able in bog-stan­dard PNG or JPG (de­pend­ing on the im­age), and JXL (ex­cept in the few cor­ner-cas­es when JXL end­ed up big­ger than JPG/PNG). Im­ages that are vis­i­ble in the HTML pages should be in­ter­laced now, ex­cept in a few cor­ner-cas­es with pix­e­lart im­ages, where they on­ly weight a few kibibytes and the in­ter­laced files end­ed up be­ing 2x the size of the non-in­ter­laced ones (in­ter­lac­ing shouldn't mat­ter much in these cas­es, even if you're on a di­alup con­nec­tion). Un­for­tu­nate­ly in­ter­lac­ing usu­al­ly blows up file sizes (ex­cept in JPEG/lossy JXL, where pro­gres­sive cod­ing usu­al­ly pro­duces small­er files), so as a com­pro­mise, linked im­ages are not in­ter­laced.

Video formats#

I al­so start­ed to play around with AV1 videos. Since I have that plyr thing now, I can have mul­ti­ple videos and the user can se­lect which one he wants to play (this is the on­ly thing that cur­rent­ly doesn't work well in the non-JS ver­sion, the brows­er just picks the first source it can play, with no op­tion to se­lect an­oth­er one).

And now comes the ug­ly part. I have to ad­mit, I had some in­valid pre­con­cep­tions. Like, FFmpeg can be used for en­cod­ing videos. But I'm get­ting ahead of my­self. There are three AV1 en­coders in FFmpeg: libaom, svt-av1 and rav1e. From a quick look it looked like the SVT-AV1 is the bet­ter one, the one is be­ing ac­tive­ly de­vel­oped and gen­er­al­ly the bet­ter, so first I checked that. Of course half of the op­tions are not avail­able on the FFmpeg com­mand line, but for­tu­nate­ly you have -svtav1-params to spec­i­fy ar­bi­trary pa­ra­me­ters... if you have FFmpeg 5. And here comes the prob­lem, be­cause FFmpeg de­vel­op­ers nev­er heard of API and ABI sta­bil­i­ty, they're con­stant­ly break­ing API with mi­nor re­leas­es, you can ex­pect the fuck­up that comes with a new ma­jor ver­sion. So even Gen­too un­sta­ble is still on FFmpeg 4.3, to avoid break­ing the whole world. In the end I back­port­ed the patch that adds -svtav1-params to FFmpeg, so I can ac­tu­al­ly use this shit. And man, it's slow. VP9 is al­so slow, but 0.3 FPS en­cod­ing re­al­ly kills my mood.

So I start look­ing around the net, find a ran­dom fo­rum mes­sage some­where (sor­ry, don't re­mem­ber where) where a guy ca­su­al­ly men­tions en­cod­ing in chunks, like it was some su­per com­mon knowl­edge. Well, maybe, I've searched for it a bit, but hard­ly found any­thing, so I just sit down and write my own script. FFmpeg has a seg­ment "mux­er", which splits the out­put in­to mul­ti­ple chunks at keyframes. Won­der­ful, ex­act­ly what I need, since svt-av1 doesn't have prop­er scene change de­tec­tion for keyframe place­ment. So the plan: con­vert the video to loss­less x264 (be­cause x264 has IMHO the best com­pro­mise be­tween en­cod­ing speed and com­pres­sion), split it at keyframes, en­code the chunks in par­al­lel, then con­cate­nate them. I quick­ly wrote a bare-bones bash script, that I copy-past­ed all over the place, but it did the job. (Lat­er I rewrote it in ru­by, be­cause if I run mul­ti­ple com­mands in par­al­lel, their out­puts will be mixed to­geth­er, and I want­ed to do some­thing about it...)

So, every­thing looks fine, let's try to use it... This pice of shit doesn't sup­port YUV444 or RGB. What the fuck­ing what? FFmpeg's wi­ki page men­tioned that svt-av1 doesn't sup­port loss­less en­cod­ing, but they for­got to men­tion this "lit­tle" short­com­ing. Any­way, this means I'll have to use libaom in­stead. And fig­ure out the dif­fer­ent op­tions, per­for­mance char­ac­ter­is­tics, and qual­i­ty of the dif­fer­ent en­coder. I'm back at square one. And even though FFmpeg has -aom-params, but many op­tions that are avail­able in the aomenc com­mand line pro­gram are not avail­able through this or oth­er FFmpeg params. Why? Be­cause libaom is a fuck­ing pa­jeet code, there's like 4 dif­fer­ent structs to store con­fig­u­ra­tion and I don't know how many ways to set it, each cov­er­ing a ran­dom sub­set of the pos­si­ble op­tions. There's a 4355 line file in the code­base which on­ly deals with con­vert­ing the op­tions from one struct to the oth­er. It's such a copy-paste mess that it hurts to look at it. But at least they use clang-format, so it ran­dom­ly changes the for­mat­ting of the re­peat­ing al­most iden­ti­cal lines, so you'll have some vari­a­tion while read­ing the re­peat­ed lines. (No, this is not a praise, this is not a fuck­ing po­em, this is sup­posed to be a func­tion­al piece of soft­ware.) And I guess they sim­ply for­got to add some op­tions to the code that han­dles the string-based op­tions (encoder_set_options in av1/av1_cx_iface.c, if you want to see some hor­ror­is­tic code)... Or maybe you're just sup­posed to use one of the oth­er 4 meth­ods to set those op­tions, but FFmpeg doesn't have that im­ple­ment­ed, so I just went with the sim­ple way and ex­tend­ed this dis­gust­ing spaghet­ti code a bit to add the op­tions I need­ed.

And I don't know how many hours of swear­ing lat­er, I had a script that cre­at­ed nice AV1 files for all the videos I need­ed. Ex­cept one thing. The segment mux­er. See, it has a -segment_time op­tion to set the length of a seg­ment, but since it can on­ly split at keyframes, I thought it works like "min­i­mum seg­ment length". WRONG! DID YOU REALLY EXPECT FFMPEG TO NOT DO SOMETHING UTTERLY RETARDED FOR ONCE? No, that's im­pos­si­ble. What it does in­stead is that it tries to cre­ate seg­ments at every n sec­onds, so for ex­am­ple with -segment_time 1, it tries to cut the first seg­ment at 1 sec­onds, the sec­ond at 2 sec­onds, the third at 3 sec­onds, even if you don't have keyframes there. So if your first keyframe is at 5 sec­onds, fol­lowed by two oth­er keyframes, you'll end up with a five sec­onds long seg­ment, then a 1 frame length seg­ment. Ba­si­cal­ly, if your -segment_time is small­er than your av­er­age keyframe dis­tance, you'll have a new chunk at each keyframe. And x264's scene de­tec­tion went crazy in some videos with flash­es and quick changes, and I end­ed up with a bunch of chunks with a few frames (many times that few num­ber of frames was 1). -keyint_min is sup­posed to set the min­i­mum keyframe in­ter­val, but it didn't work ei­ther. There's a -min_seg_duration op­tion for the segment mux­er, but my FFmpeg ver­sion was too old to have it. Plus I'd rather set the num­ber of frames, not time (I use vari­able fram­er­ate, so time is not a use­ful unit). The on­ly thing that works with frames is the -segment_frames op­tions, but it re­quires me to sup­ply all the frame num­bers I want to split the video in ad­vance. I have no fuck­ing idea where do I want to split the video un­til I've en­cod­ed it to H264. Maybe I could have writ­ten yet an­oth­er workaround, to write a sin­gle H264 file first, dump the keyframes with ffmpeg, then fig­ure out where I need those keyframes, but at this point I was like, hav­ing one more or less patch doesn't make a dif­fer­ence. I guess I should start open­sourc­ing this shit, maybe some­one will find some­thing use­ful here.

Are we done yet? Nah. I'm break­ing the chronol­o­gy here, but dur­ing the process I've re­al­ized that AV1 should sup­port RGB video, so maybe I should use that in­stead of the lossy YUV con­ver­sion. Chromi­um and Fire­fox played back RGB videos fine, but Pale Moon (and some oth­er Fire­fox forks, like Wa­ter­fox Clas­sic) didn't. Be­cause Pale Moon is the brows­er I care about the most, in the end I fixed it to sup­port RGB videos (and a bit more). I don't want to write too much about this, it was merged up­stream, and should be part of the next re­lease. Not sure about oth­er browsers out there, Wa­ter­fox Clas­sic looks dead, and oth­ers I test­ed ei­ther didn't sup­port AV1 at all or they sup­port­ed RGB. (Note, in Pale Moon, you need media.av1.enabled set to true in about:config for the time be­ing.)

Video thumbnails#

One nifty fea­ture I re­al­ly like about this new JS play­er crap is that you can have thumb­nails as you hov­er over the scroll bar. The im­ple­men­ta­tion... not much. The plyr doc­u­men­ta­tion doesn't re­al­ly men­tion any way to do this, ffm­peg should be able to do this, but it's less than ide­al. It's tile fil­ter re­quires you to spec­i­fy the num­ber of rows and columns in ad­vance, which means you need to know the num­ber of screen­shots in ad­vance. So first I cre­at­ed an­oth­er ru­by script, that takes tells FFmpeg to out­put thumb­nails to image2pipe, and it cre­ates the tiled im­age with the cor­rect size at the end. But then I al­so re­al­ized, that while us­ing scene de­tec­tion to cre­ate thumb­nails sounds good in the­o­ry, but if I try to ap­ply it to the videos I've up­loaded here, I end up with ei­ther 1 or 386 screen­shots per video. (Ac­tu­al­ly makes sense, the game­play videos hard­ly have any scene change, while eu­pho­ria's and tasog­a­re's in­tro videos gen­er­ate shit­loads of "scene changes".) So in the end I went with sim­ply cre­at­ing screen­shots every N sec­onds, but for that I need to know the length of the video (I want screen­shots every 5 sec­onds, but I want to make sure that every video has at least 10 and at most 100 screen­shots, so I need to ad­just the in­ter­val based on the video length). Iron­i­cal­ly, this way I ac­tu­al­ly don't need this dy­nam­ic tile siz­ing (as I know the num­ber of screen­shots I'll have in ad­vance), but I kept it. Al­so, I'm not sat­is­fied with the cur­rent sit­u­a­tion, so it might change in the fu­ture, and in this case that code might be use­ful again. (For ex­am­ple, the last ~2.5 min­utes of the tasog­a­re's in­tro video is a still im­age, yet it now has that sin­gle im­age thumb­nailed 33 times.)

What's next?#

Dun­no. There are 19327849233292 things I want to im­prove, but I usu­al­ly end up bored quick and stop. I re­al­ly don't like the video play­er, so I might do some­thing about it. I've writ­ten some JS code by hand, but I'm not sure whether I want to con­tin­ue like this. JS is a fuck­ing abom­i­na­tion, I'd like some­thing more sane, maybe with sta­t­ic type check­ing. I might look in­to tran­spilers, but in that case I'll com­pli­cate the build process of this site even more, and they usu­al­ly gen­er­ate shit­loads of bloat. Al­so, it doesn't help that most JS tran­spilers ex­pect you to have a full-blown node­js ready, and I re­al­ly don't want to in­te­grate all that crap in­to my nanoc site.