An interactive explorer for Benford's Law across real datasets

(vatsalbakshi.com)

37 points | by dingobabies 11 hours ago ago

13 comments

  • jmpman 10 hours ago ago

    I'd recommend allowing 2 digit exploration. I've used it in the past when analyzing hard drive failure logical block addresses.

    • nextaccountic 8 hours ago ago

      benford's law is used to detect whether data is faked

      in which ways would the list of hdd bad blocks be faked?

      • jmpman 28 minutes ago ago

        You can use it to find anomalies in data sets. I'd read somewhere that Starbucks coffee prices stick out on financial audits using benfords law, because they're over represented.

        In my scenario, I simply had access to a massive data set of hard drive sector failure LBAs, and had just read about benford's law on Slashdot (back in the day), and was skeptical that benford's law would actually fit this data.... so I tried it. And the data fit benford's law perfectly - with a single digit. When I tried it with 2 digits, and again, it fit perfectly - except there were two spikes which stood out. Curious, I started digging in... it was an interesting failure mode.

      • wonnage 6 hours ago ago

        It can also be tried for general anomaly detection https://dl.acm.org/doi/fullHtml/10.1145/3624062.3624121

  • yellow_postit 9 hours ago ago

    Neat! Benford’s Law was the first topic I dove into in undergrad math that got a minor publication. Given how well known it is for forensic accounting I’ve always wanted to look into convictions and see if the “average” fraudster has wised up and produces more realistic distributions.

    • cwmoore 3 hours ago ago

      Could look up convictions and see how any county courthouse delivers sentences, Bendord’s Law isn’t just for accounting, it’s for anything numeric.

    • nextaccountic 8 hours ago ago

      i suppose that nowadays analysts have more sophisticated tests?

      in any case, for any set of statistical tests, it's relatively trivial to produce data that passes all of them

  • jboggan 8 hours ago ago

    I once did an application of Benford's Law to USDT transactions between crypto exchanges, which seemed to indicate some exchanges had mostly "organic" transactions and a handful of exchanges seemed to have heavy transaction volume of seemingly-random but not really random amounts, indicating some level of wash trading on those exchanges.

  • sorokod 2 hours ago ago

    Interesting topic, shame about the LLM phrasing.

  • cwmoore 10 hours ago ago

    Interesting that it was first discovered with noticing the “garden path” in the front pages of a book of logarithm tables (in 1881).

  • deanalyzer 7 hours ago ago

    I learned about Benford's law over a decade ago, and I always found it beautiful and elegant. But surely, fraudsters have become more sophisticated by now. I wonder if you asked an AI to commit fraud, if it would be clever enough to avoid such mistakes.

  • anArbitraryOne 9 hours ago ago

    It's interesting how all base 10 numbers identify as non binary

  • SV_BubbleTime 7 hours ago ago

    I wonder if it using this would help disprove election irregularities?