You Too Could Have Made Curl

(daniel.haxx.se)

136 points | by sebg a day ago ago

99 comments

  • hodgesrm a day ago ago

    I saw this talk in person. It was delightful. My favorite part was the stories of helping random people who contacted him to help fix things like car electronics. They found his email from the OSS license acknowledgments, which apparently are accessible to ordinary users.

    (As they should be.)

    • oefrha a day ago ago

      Well he also got "I will slaughter you" through the exposure of his email in OSS license acknowledgements, so it's not always fun after all.

      https://daniel.haxx.se/blog/2021/02/19/i-will-slaughter-you/

      • tivert 19 hours ago ago

        After reading that I thought the guy who sent that email must have been crazy, and it turns out he was: https://daniel.haxx.se/blog/2021/08/09/nocais-apology/:

        > I had to retire from my career do to schizophrenia. Again, I should have not let my delusions go to the point they did nor should I have acted the way it does. My illness doesn’t detract from the rashness of my actions.

        There a lot of technical word salad there, like "When I say hacked, I lost every device. They rooted my charge arbitrator, I was bios bonded and I basically lost every document and all my software."

    • wslh a day ago ago

      [flagged]

  • trustno2 a day ago ago

    It seems like getting projectors to run is really the hardest problem in computer science.

    • jimbokun a day ago ago

      I've been in Computer Science graduate courses and lectures in one of the top programs in the US where it seemed like every week the first 5 to 10 minutes was spent trying to get the projector to work with the lecturers laptop.

      • zikduruqe a day ago ago

        What amazes me on conference calls is how no one can explain audio issues. Really, really smart people on a call that can explain registers, cryptography, kernel tuning.

        Sounds like you are having network issues/bit errors? You are breaking up.

        Sounds like your bluetooth is having issues? You are breaking up.

        Sounds like they are mobile, and in a weak service area? You are breaking up.

        • atoav a day ago ago

          As a person who is an audio engineer, dsp programmer and supporting such systems professionally: That is because the breaking up for the most part sounds very much the same. It is a CODEC specialized for low bitrate transmission going into low bitrate compressions, packets being dropped entirely or arriving with a delay etc.

          Who knows where such a thing comes from? It could be any number of things on a huge technological stack distributed over a large geographical area.

          A domain expert should of course be able to distinguish various bad conditions by ear, e.g. clipping, saturation, wromg microphone distances/orientation, signal interferences, hum, bad grounding etc.

          But not all people are good at analytical hearing regardless of them being engineers or not. That is why people pay the likes of me.

        • yjftsjthsd-h a day ago ago

          Are those things distinguishable? I thought people say you're breaking up because they only see the symptom of audio cutting out and have no way to distinguish what the cause is.

          • atoav a day ago ago

            Mediatec professional here: hard/impossible to distinguish especially, because knowing precisely which signal processing your video-conferencing solution might be doing under which conditions and in which version is even hard to know for people who analyze one version of software in a lab environment.

            That is like a relative saying you are not an IT expert because you don't know immidiately what the distinct root cause of "the screen being black" is.

        • tristor a day ago ago

          This is mostly a consequence of the complexity of all the abstraction layers between "conference call" and "physical routed and switched connection". At best you might be able to identify that the visible/audible symptom is due to packet loss, but proving a root cause of the packet loss is extremely difficult. Through some tooling, like ThousandEyes (which I work on), you might be able to identify the hop in the path that's causing that forwarding loss, but unless you have access to that device it'd be impossible to prove exactly /why/ it has forwarding loss.

          Any type of problem like that ultimately becomes a "5 Whys?" kind of troubleshooting to get to a real root cause, and from a end-user device you generally don't have the necessary access or data to answer more than 2-3 layers of abstraction.

        • SpicyLemonZest a day ago ago

          I try to be more specific, but there's just so much going into modern sound processing. Told a guy once his bluetooth connection was failing, but the real issue was that he had a plane taking off overhead and the VC software was noise-cancelling everything.

      • mlyle a day ago ago

        I just have a ChromeCast in each room that I lecture in. Easy to bounce a tab over to the big screen, and it turns on the TV/projector. It also helps prevent notification or other screen sharing related mishaps.

        My biggest complaints about this approach are that I can't easily see speaker notes (I have a fiddly workaround that I can use if I need it, but it would be nice if Google Slides would support the ChromeCast use case a little better..) and that the TVs/projectors tend to screw around for 10-15 seconds before automatically choosing the right source.

        • dvngnt_ a day ago ago

          would phone work?

          • mlyle 20 hours ago ago

            I know I can open other tabs to look at the presentation, so I assume it would. But it's not good optics for me to have a phone out when I ask my students not to do so.

            I can get a speaker notes and a normal presentation tab open and cast one of them, but they all jump around and do the wrong thing on my screen (wanting to maximize, etc) and require a lot of coercion; too much time to start a normal lecture.

    • Maken a day ago ago

      Harder than getting printers to print?

      • eka1 a day ago ago

        Typically, you aren't trying to print in front of a large audience

      • trustno2 a day ago ago

        Hey that's what we get CUPS auto-discovery for.

      • high_na_euv a day ago ago

        It is unreal how tech as mature as printers is so fucking unreliable

        • makeitdouble a day ago ago

          Arguably it still hasn't matured yet. Or at least it's nowhere near to be a solved problem and we'll probably sidestep it before ever coming up with a real solution.

          I'd put it in the same bucket as TODO lists. After centuries we're still not set.

        • bigstrat2003 a day ago ago

          Depends on the printer. I still remember the LaserJet 4000 series with great fondness. They were absolute tanks that almost never broke. You did have to replace wear items like rollers, but that was it.

      • mschuster91 a day ago ago

        At least printers give you some sort of error message.

        EDID negotiation or signal integrity issues in HDMI, good luck diagnosing actual causes without highly specialized (and expensive...) equipment.

    • thefaux a day ago ago

      This feels like such a fixable problem. Presenters should be required to show up early and test the av or send the conference their slides in advance.

      • ryandrake a day ago ago

        The burden shouldn't be on users. Why are computer manufacturers putting out a hardware/OS combination that can't work reliably with an external display? Don't they even test this common use case?

        Even my Mac (which, to Apple's credit, does work 100% reliably with projectors) still struggles when I plug in multiple external monitors, blanking the screen, turning on one external display, blanking them both, turning on the other one, and so on. This is a manufacturer who normally accepts only a smooth, polished user experience, and they still can't get it perfect. What chance do Lenovo or similar garbage-tier plastic box manufacturers have of getting it right?

    • rqtwteye a day ago ago

      Right after getting the screen layout right when you move between home office and work office :-)

      • a1o a day ago ago

        Seriously should be possible to save those in a layouts pane to force adjust the monitors

    • strunz a day ago ago

      Say what you want about Apple but I've never had a problem printing or connecting a monitor/project (as long as I have the correct dongle :P )

    • netsharc a day ago ago

      Meanwhile, "this scratchy audio is fine!".

      I guess it's something one can throw AI at. It probably doesn't need GPUs, just some audio filters.

    • jdboyd a day ago ago

      I've had good luck slapping an Extron IN-1606 in front of a projector to make all the compatibility problems go away.

    • troupo a day ago ago

      I used to joke it was ironic how people in management would always have their presentations up and running almost immediately while programmers seemed to always struggle with projectors.

      These days however nothing seems to work for anyone.

      • RcouF1uZ4gsC a day ago ago

        Management is typically running a stock Windows Dell/HP/Lenovo or a Mac.

        Programmers are running some custom Linux distro with a custom window manager and other customizations.

        • asveikau a day ago ago

          The window manager won't matter at all in the HDMI/EDID negotiation with the projector.

          Windows vs the X server may make a difference. But which X11 WM you use should be irrelevant.

          • lelandbatey a day ago ago

            Technically yes, but since the WM usually comes as part of a desktop environment that also includes utilities and daemons which help with that sort of thing, to many folks "my WM makes a big difference in how easy it is to set up external monitors" is often true.

            It drives me crazy that there's not a long running named process in e.g. GNOME the DE that if I run will cause my fn-volume keys on my laptop keyboard to change the volume any WM, the same as when I'm running GNOME? Why isn't that piece separated out? Or at least, where is the code so I can separate it out?

            Questions and experiences like that cause many to conflate behavior and experience with WM/DE.

    • mbonnet a day ago ago

      A/V is the final frontier, not P ?= NP

    • AndyMcConachie a day ago ago

      I once helped Andrew Tannenbaum get his laptop to work with the projector in the room. His presentation was only slightly delayed because of it, but the irony ...

  • silvestrov a day ago ago

    The problem is not to make a tool like curl.

    The problem is the marketing and getting people to use it.

    You can easily make the best product and still not being able to get the word out there.

    • bityard a day ago ago

      I don't think Daniel "marketed" curl?

      I couldn't say when curl was released without looking it up, but I know it's been around for at least a couple of decades. There were not many command-line tools for dealing with web stuff, and curl got picked up by most Linux distributions, the BSDs, and various embedded things because it was the only tool that did what it did, as well as it did. And then it became famous essentially for being ubiquitous, like bash or vim.

      "Marketing" seems to have had very little (if anything at all) to do with it.

      • 0cf8612b2e1e a day ago ago

        Best-in-class vs First-in-class (not that I have any criticisms of curl). Sometimes just being first is all it takes to win.

        • alisonatwork a day ago ago

          I am pretty sure wget existed before curl, insofar as I remember people back in the day talking about curl being a newer/better version of wget. From my limited usage it didn't seem like a better version because I had to remember a bunch of switches to get it to just download the file which wget already did out of the box, but somehow it became the industry standard anyway, perhaps because of the more permissive license. I don't have any particular complaints with it either, although I still always forget the switches I need to do to make it download a file instead of print a bunch of stuff to stdout. Oh well.

          • mixmastamyk a day ago ago

            They overlap but wget focuses on downloading by default, while curl is more general purpose. Once I found httpie I didn’t really use either anymore, however.

        • TZubiri a day ago ago

          And being consistent too.

          • jeanlucas a day ago ago

            I call that "last man standing" strategy, works really well in mature SaaS markets as well.

    • JeremyNT a day ago ago

      If curl was created today? Sure.

      But curl was something new when it came out, and it truly filled a void both as a cli tool and as a library. It needed no marketing. We needed it.

      It's hard now to imagine a world without a tool like curl, but quite a few of us are old enough to remember it still.

    • hnthrow289570 a day ago ago

      Word of mouth can still help you, but you at least need to find the community where people have the shared problem, so their recommendations have weight

      Overcoming the initial "why should I even look at this" to me is the hardest part

    • ww520 a day ago ago

      Amen. Marketing is so important for software, even for OS software.

    • tootie a day ago ago

      There was a bit of PR "war" between curl and wget a long time ago. At some point curl just became everyone's goto and even the reference implementation for http.

      • masto a day ago ago

        Many moons ago, I was in the small ISP business, and I discovered the aftermath of an attempted script kiddie hack on one of our servers. When I examined the logs I realized we were extremely vulnerable to the remote code execution exploit but had been completely saved by two things: they kept trying to use curl to install the payload, but we only had wget installed; and their scripts were extremely Linux-centric but we were using FreeBSD.

        • bityard a day ago ago

          That's funny, I remember the exact same thing. It was either Joomla or Wordpress for us, and we were saved only by virtue of having everything in jails and those jails having a very limited (and un-Linux-like) userland.

      • mijoharas a day ago ago

        Curl's success has more to do with it's quality (and ubiquity) as a C library. I think the curl cli somewhat got "taken along for a ride" with all the improvements that curl lib had by becoming the de-facto standard http library.

        Because the library gets ported to every new platform anyways (it's what people use so will be the first to be ported to a new architecture or whatever) so the cli gets support for everything new "for free" so can outcompete wget (because it's always _there_ and works the same).

        Now, there's also the fact that the curl cli is a fantastic piece of software, but in terms of features I don't think there's that much between curl and wget for simple cli usecases (but I still use curl).

      • tannhaeuser a day ago ago

        wget (and htrack) is able to parse downloaded HTML and CSS for scanning additional URLs to fetch recursively and mirror entire sites that way, while curl is "just" a very very complete HTTP client.

        Though wget isn't perfect for mirroring, as it will typically download large amounts of resources redundantly only differing in URL query params when those refer to the same content, such as on typical WeirdPress sites with comment links. Ideal would be an HTTP client with http/3, auth token, and keepalive support etc. based on libcurl that can be customized, such as via "href handlers" triggered by event-driven markup parsers for SGML (though CSS and JS imports need special treatment).

      • bigstrat2003 a day ago ago

        Curl certainly isn't my go-to, because it's so complicated. For most uses I can just "wget http://blahblah" and it just works. With curl I have to look up the flags to save the file, etc. Curl is certainly more powerful, but that power comes with a steep usability loss for the common case.

        • aftbit a day ago ago

          "curl http://blahblah > blahblah.html"

          What other common cases take flags on curl but not wget?

          • electroly a day ago ago

            -L (follow redirects) very commonly appears in curl commands, whereas it's the default in wget without needing a flag.

      • taneq a day ago ago

        I never did really figure out why curl became a thing when we’d been wget to do that since 2001.

    • ShanAIDev a day ago ago

      It seems like the first-mover advantage really plays a significant role here.

  • jeanlucas a day ago ago

    Daniel is a unique OSS builder. We asked him to give a talk to our community and he accepted, no questions asked, just showed up :-)

    His commitment to spread the word of curl and OSS is really admirable.

  • mschuster91 a day ago ago

    > Several people eventually got involved, things were rebooted multiple times, cables were yanked and replugged in again. First after I installed arandr and forced-updated the resolution of my HDMI output to 1920×1080 the projector would suddenly show my presentation. (Later on I was told that people had the same problem in this room the day before…)

    That's why it is a good idea if you're a presenter to always have a Decimator [1] or its cheaper and less-capable alternative from BMD [2] in your pocket. No matter what, both of them will make sure you can get at least a signal to any sink with an HDMI or SDI port. Even 20 years after EDID, negotiation issues reign supreme...

    Personally, when I'm responsible for inhouse conferences, I always have the venue place a Decimator at the presenter desk and run the cable to the projector using SDI - particularly MacBooks and their USB-C dongles tend to have issues with cable lengths above 5-10 meters, I'd guess too low voltage swing.

    I literally never ran into incompatibility issues with this setup, only issue I had was some dumbass thinking they could unplug the Decimator to charge their phone. Additionally, SDI makes it very easy to install a split-off run to a central recording site, even if it's 100 meters of cable run away, or a split screen at the presenter's feet. No negotiation issues, everything Just F...ing Works.

    [1] https://decimator.com/Products/MiniConverters/12G-CROSS/12G-...

    [2] https://www.blackmagicdesign.com/de/products/miniconverters/...

    • lqet a day ago ago

      Interesting, what exactly does the Decimator do?

      (Side note: I don't think it is a particularly clever idea of Decimator Design to fill their logo and homepage with images of large spiders. They are effectively eliminating around 10% of the population with arachnophobia as customers, and are making the experience of visiting their website slightly uncomfortable for large parts of the remaining population.)

      • Scaevolus a day ago ago

        Decimator Design excluding 10% of the population is a historically accurate decimation.

      • mschuster91 a day ago ago

        > Interesting, what exactly does the Decimator do?

        A multitude of things, depending on how you configure it:

        - signal boosting of low-voltage or otherwise problematic HDMI or SDI signals, including reclocking. That's imposing virtually no latency and if you're running in a TV broadcast environment with centralized clocking it makes work easier for the downstream mixer. Primarily, it avoids issues with mis-designed sources such as cheap-ass adapter dongles that have signal integrity issues - as long as it's "good enough" for the Decimator to recognize it will output a clean and high-strength signal.

        - format conversion from HDMI/DVI to SDI and vice versa. That imposes a tiny bit of latency and allows the usage of "consumer" equipment with broadcast equipment, or if you use two Decimators linked by SDI, you can "just" go for far longer cable runs than you could ever do with HDMI - up to 100m, in practice way more.

        - up and downscaling as well as framerate conversions. This is where things really get interesting - no matter what kind of source and sink you use or what their limitations are, the scaler engine can make them talk. That includes color space conversions and bandwidth as well, and it's particularly useful if you, say, have a 12G SDI link from the source to the sink, and you want to use an older display for additional monitoring. Splice a Decimator in there and it will output the full quality signal on one output while providing a scaled-down signal on the second output.

      • wruza a day ago ago

        Yeah, that’s the perfect case of a good product overrun by a questionable design idea. And its name sounds 13yo too, tbh.

      • draw_down a day ago ago

        [dead]

      • a day ago ago
        [deleted]
    • stephen_g a day ago ago

      Yes, SDI was just designed for this kind of thing (moving video long distances between cameras in studios, tape decks and patch bays in technical rooms, vision switches and capture equipment in editing bays, etc.), whereas HDMI was only ever designed for connections of a few metres (and then active optical cables had to be made to try and fix the mess)…

    • anthonyeden a day ago ago

      I agree. I personally leave my Decimator MD-HX’s in ‘Free Run’ scaling mode at my preferred resolution. The in-built ‘no signal colour’ of my choosing helps me troubleshoot the problem in a signal chain quickly.

      Worth mentioning for the novices that SDI does not do HDCP, so you can’t pass copy-protected content via this setup without workarounds.

      Of course, it really should be up to the conference AV suppliers to organise all this, not the presenters themselves.

      • mschuster91 a day ago ago

        > Worth mentioning for the novices that SDI does not do HDCP, so you can’t pass copy-protected content via this setup without workarounds.

        Well, the workaround is a 10$ HDCP stripper from ebay. Easy enough. (And I think that both Decimators and BMD converters also strip HDCP, but don't quote me on that)

        > Of course, it really should be up to the conference AV suppliers to organise all this, not the presenters themselves.

        Well, conference setups tend to be a hit and miss IME. Some where the venue and the conference is run by dedicated professionals, you don't even have to tell them to place Decimators and SDI cable runs, they'll come in with these from the start in their offer... but a lot of smaller venues and conferences (especially those ran by volunteers / NGOs) don't have that knowledge and just place some random home cinema projector together with a 20 meter HDMI cable. Corporate offices are just as bad IME, I've never seen a corp conference room with anything but HDMI or, in the worst cases, VGA. Corporate just doesn't give a flying fuck unless they got a dedicated team and a proper budget working on conference room setups...

    • Arelius a day ago ago

      It looks like the MD-HX is what you want for this use case?

      I do a bunch of international travel, and the HD-LX seems ideal sized, but it seems like it doesn't do most of the useful functions?

      • mschuster91 21 hours ago ago

        It's "only" 3G-SDI so FullHD only.

  • shark1 20 hours ago ago

    The man built a software used by billions and still gets nervous talking in public :)

    It was a delightful presentation.

  • tempfile a day ago ago

    arandr mentioned! I use it all the time. Never had an issue.

  • ketanmaheshwari a day ago ago

    "Very few things are an immediate success". Wise words.

  • a day ago ago
    [deleted]
  • kunley a day ago ago

    [flagged]

    • marginalia_nu a day ago ago

      I think this is too an extreme a take.

      One the one hand, yes, you can absolutely amuse your life into ruins. 24/7 access to fun is the great affliction of our times. Video games are part of it, but so is netflix and social media. If you're spending 20 hours a week partaking in any of these things, that's 20 hours you aren't spending building stuff or honing your skills. It really does add up over time.

      On the other hand, you do not need to join the desert fathers to be productive, in fact, taking a moment every now and then to unwind and do something mindless and enjoyable instead helps you last the course, and a break from work can be very helpful in allowing your brain to process what you're working on, often with new inspiration and ideas as a result.

      The best approach is to structure your life a bit, instead of having this all-or-nothing mentality where you either always work or constantly binge on amusements, I have a particular day of the week I play video games and do stuff like that. Works great.

      • marxisttemp a day ago ago

        I’ve found that the periodic passive money-making activities in Grand Theft Auto Online are the perfect thing while I’m coding; every hour or so I’m reminded to empty various safes and acquire various items that are on cooldowns, which usually takes around 5 minutes and is a perfect little break to ponder whatever I’m working on.

      • hnthrow289570 a day ago ago

        >that's 20 hours you aren't spending building stuff or honing your skills

        My problem is it always seems like I'm building skills to help some capitalist be more rich or make more money. If I want to build something to contribute to a community sans profit, there's the burdens of popularity if I'm successful.

        I think people are afraid to admit that amusement is what brings them fulfillment. Having to work on something or build more skills sounds similar to a protestant work ethic to me, which was much more important when the essentials of living were not as assured as it is today (or you need to fit into certain social circles today).

        I suppose it helps your personal character and having more things to talk about, but I don't know if worrying whether or not a majority of people I'd meet find me interesting because of my skill(s) is also a good selling point.

    • oh_my_goodness a day ago ago

      For some of us, it's no-gaming or way-too-much-gaming! It's also possible that the author might be employing a bit of the (deprecated) rhetorical device "irony".

    • nbaugh1 a day ago ago

      Why? Is the entire gaming industry just a big waste of time?

      It feels a bit like saying "I never watch TV", "I never listen to music", or "I never read fiction" - "I only ever code, I am purely productive."

    • udev4096 a day ago ago

      I don't think computer games are bad. In fact, I would say most of the people who grew up playing games wondered, at some point, on how to make those games which eventually helped them discover different computer topics

      • jumping_frog a day ago ago

        This is also the reason why game industry is very evil. That steady supply of youngsters trying to prove themselves on their teenage profession choice are taken advantage of. If they pivoted to other software domains, I can understand.

      • marginalia_nu a day ago ago

        It's probably quite complex.

        I remember when I was in high school when World of Warcraft dropped, like half the guys in my class got super hooked and would stay up late every night and phone in the school work every day. Their grades just never recovered from WoW, even some of the really bright ones graduated with barely passing grades.

        Though admittedly WoW was a bit of a killer cocktail, because of the social pressure to partake in raids. That kind of game was also so new, like yeah there had been MMOs before, but nothing like that, so the "WoW effect" may admittedly be a relatively unique event in gaming history.

    • Insanity a day ago ago

      What does it matter?

      • wruza a day ago ago

        Probably a part of I-never-play-games cult. Seems to mean something for them.

        • DiggyJohnson a day ago ago

          > Probably ... cult

          You simply have a different perspective, and are overinflating their position.

        • kunley a day ago ago

          [flagged]

          • dijit a day ago ago

            > Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.

            https://news.ycombinator.com/newsguidelines.html

            • kunley a day ago ago

              I can comply if the same rule is applied to the passive-aggresive cult remark above. But if it's not, I say no to double standards.

              In fact, my original remark didn't (yet) revealed much details and reasoning about my view on gaming, but of course it suggested that it's quite a topic. I would expect that it could be an introduction to a civilized talk about the topic - while accusing me of being in a cult is certainly not.

              That's why also I said the guy was triggered. If he was not, the correct reply to my remark, even if he disagreed with the premise, would be sth like "what do you mean"

              • wruza a day ago ago

                I apologize and s/cult/club/. That said, I comment as I feel reasonable and don’t try to meet expectations on conversation structure that much. You never replied to anyone itt who did, despite them never mentioning anything remotely aggressive.

                • kunley a day ago ago

                  That's fair, thank you and I accept this.

                  And you know, "club" is a totally different word. I would be even kind of proud of having such a label :)

                  As for my lack of broader answer, didn't yet have time for it and I would really like to address few things, and then this subthread happened, well, let's move on..

          • wruza a day ago ago

            I’m not triggered at all. I don’t even understand what that means for them cause every interaction ended in no explanation besides some weird remarks.

            • DiggyJohnson a day ago ago

              Not GP, but it seems like you are assuming the worst in the author simply because they are outspoken about not playing video games. That's what eliciting these reactions.

      • a day ago ago
        [deleted]