WHY2025: How to become your own ISP [video]

(media.ccc.de)

209 points | by exiguus 3 days ago ago

53 comments

  • abound 3 days ago ago

    I got my own ASN a few months ago (and am in the three+ year waiting list for an IPv4 block), I've been thinking about trying to become a rural ISP in my area, so this is very timely for me.

    • theideaofcoffee 3 days ago ago

      If you have serious intentions of starting an ISP, I'd recommend beginning conversations with a few transit providers right away, feeling out rates and commit terms. Armed with some market info and contacts there, start to look at v4 auctions via [0] or similar so you can jump the line, though you'll have to pay for the privilege. You probably won't be able to transfer blocks into your org until you have commits from one or more upstreams (I know ARIN, and I'm assuming you're in north america, has some more stringent reqs in terms of overall usage within a specified time period, so choose the auction size appropriately [1]). You may also want to consider taking a block from your transit, they will often reassign a small prefix out of their larger holdings for customer use. You can often use that as justification for transferring future blocks.

      After transit, start to look at facilities to host your equipment, 'cause you'll need to demarc somewhere and hand off to your transit as well.

      Lots and lots of details to get right, but I personally think it's a lot of fun.

      [0] https://auctions.ipv4.global/

      [1] https://www.arin.net/participate/policy/nrpm/#eight5

      • nine_k 3 days ago ago

        Regarding IPv4 auctions: does a small ISP even need a pool of IPv4 addresses? Mobile providers, such as T-Mobile, happily run IPv6-only networks, and provide 4-6-4 address trssncoding to access IPv4-only sites (hello GitHub).

        Would this be more expensive for a small ISP than paying for /26, or whatever pool size is practical?

        • theideaofcoffee 3 days ago ago

          I believe you're referring to 464XLAT (RFC6877 [0]) and yeah, you wouldn't -need- to have any ipv4 stack at all internally (except at the very edge of the network to number the PLAT devices [1]), but I believe it would cause a higher support burden for the nascent ISP than it would relieve by not having to run v4 and v6 together. There may be devices a customer owns that just doesn't support v6, or has weird bugs that would be a show-stopper for them. Should everything, ideally, be supporting IPv6? Yes, of course. Does v6 work seamlessly in all situations? Absolutely not.

          I think the need to run a dual-stacked network, especially one that serves a wider customer base will be required for years, perhaps a decade or more, to come. If we were able to control every device and know it has a well-behaved v6 stack, then it might be a different story (which might be the case of T-Mo, as handset variations are limited in scope and well-defined in that scope, and behavior, mostly). But we still need v4 somewhere, and will continue to need it until the bulk of the internet is migrated.

          I've had the luxury in the past of having complete control over the devices running in a v6-only network and even then it was a struggle to confidently say that everything had perfect connectivity at all times, even with tricks like 464XLAT or SIIT [2] at the edge. I can't imagine the pain of a network with heterogeneous customer devices running v6 stacks of varying quality.

          Anyway, lots of words to say that it theoretically could be done, I just don't see it successfully being done with all of the variations in a consumer-facing network. The gulf between theoretical and the practical implementation is vast. Personally, the going rate for a block of /24 or /23 or whatever size is a small price to pay for compatibility.

          [0] https://datatracker.ietf.org/doc/html/rfc6877

          [1] https://datatracker.ietf.org/doc/html/rfc6877#section-2

          [2] Stateless IP/ICMP Translation

          • xcrunner529 2 days ago ago

            Lots of other countries have ISPs that are 6 only today. It’s a privilege at this point that the US doesn’t have to care as much. You don’t need it.

        • ta1243 3 days ago ago

          You'll need at least one ipv4 address to hide your customers behind when they access an ipv4 network.

          An ipv4 address costs $30 to purchase at a /24 level, less in larger amounts.

          If you are providing service to a customer that's $2.50 a month for a year.

        • zajio1am 3 days ago ago

          You don't. You can get some IP addresses from your upstream, use private IPs inside your network, and do CG-NAT on the border router.

        • jeroenhd 3 days ago ago

          Using IPv6 internally and using CGNAT or whatever translation layer you'd prefer for external IPv4 access would be the cheapest solution, but I think many of us would like to run dual stack. I can understand why someone would like their own larger IPv4 range when starting a new ISP.

        • 3 days ago ago
          [deleted]
      • abound 3 days ago ago

        Thanks for the tips! Being in a rural area, there's only a few colocation facilities within ~50 miles, and I need to reach out to them to see who they're connected to. So far, I've only seen that Cogent has a presence here, but the internet doesn't have good things to say about them

        • bc569a80a344f9c 3 days ago ago

          > I've only seen that Cogent has a presence here, but the internet doesn't have good things to say about them

          They’re good enough and they’re dirt cheap. Pricing really matters since you pass savings on to your customers. Vermont Telecom, as an example of a reasonably sized regional ISP with thousands of customers, uses Cogent as their primary upstream.

          I wouldn’t fall into the trap of trying to build something out using hardware or upstream providers that people on Internet forums that don’t have a financial stake in making an ISP work financially and operationally approve of.

        • toast0 3 days ago ago

          I think Cogent has earned their vibes, but if they're all you can get near you, they're ok enough to get started with.

          As you get bigger, you can put a router somewhere (or two somewheres) with cogent and more options and transport through cogent to get back to your service area. Looking at your profile, I think if you can get traffic to Denver, you should have more options there.

        • inemesitaffia 3 days ago ago

          Cogent can sell you transport to an IX where you'll get lots of traffic for free.

          You don't have to buy IP transit from who sells you physical connectivity.

          I've heard they have good connectivity with Comcast.

          Consider one of their customers, FDC.

        • theideaofcoffee 3 days ago ago

          Cogent is fine for most purposes. They have some odd choices upstream at times but for a smaller project, it shouldn’t be a problem. Later on you could look at a dedicated wave or ring (or tunnel, lots of options!) to another facility that has more diversity and peer through that, but make it work first!

        • 3 days ago ago
          [deleted]
    • esseph 3 days ago ago

      If you want someone to bounce ideas off of, I've been involved in that space for a very long time and could probably answer a lot of questions with real world experience, regardless of the technology used.

      • abound 3 days ago ago

        I sincerely appreciate the offer, and would love to take you up on it! My email is available from my profile (via my website), or feel free to grab some time: https://cal.bsprague.com/meet

        I've mostly looked at wireless (we're in a valley) and fiber

        • jrklabs_com 3 days ago ago

          I'm likely heading for the same destination as we're building a new house in a rural area that has limited Internet options. If you're open to it, I would love to join any meetings that you may have regarding building out a new WISP. I do have one wireless bridge live now that was my proof of concept.

          • esseph 2 days ago ago

            Didn't see any contact info in your profile, but if you send an email to: jr331288726745 at nervous.ink I'll respond.

  • rmoriz 3 days ago ago

    LIR and resource fees at RIPE are too damn high compared to ARIN. Europe is hurting itself by punishing small entities.

    • 47282847 3 days ago ago

      Annual LIR fees at RIPE are around 2000€, and includes trainings and tickets to meetings. If anything, this serves as a filter for economically unreasonable ventures. Compared to other expenses, even if run purely on volunteer basis (meaning you are donating your time), this seems easily doable.

      • weitendorf 3 days ago ago

        I've been doing a moderate amount of research on getting an ASN and ipv4/6 blocks so I can BYOIP and host third-party services without being locked into the cloud provider I was using at the time the third-party configured DNS. That has led me down various rabbit holes in which I started learning how the Internet actually works.

        IMO the Internet actually sucks ass

        Why is there so much bureaucracy and cost involved for someone to own an IP address? I should be able to connect to the network and acquire an IP address as easily as I can buy a merckle-tree-backed pointer to an IPFS image, or vote in a US election. Why do I have to pay hundreds of dollars for Internet Numbers conjured from thin air by a US nonprofit to be resold by a RIR? How fucking moronic is it that IPV4 was created with substantially less capacity than there were humans on Earth, got adopted, wasn't immediately fixed or abandoned once it became obvious that the Internet would be used globally, was irresponsibly allocated, introduced various unofficial but consequential practices (eg NAT), ran out and got expensive, and STILL is widely used alongside ipv6.

        What is the point of having a centralized system for governance centered around ICANN/IANA when they are so wildly inefficient and incapable of governing? Fuck 2000€ these are freaking made up numbers that I should be able to buy for pennies with an email address, government ID, and credit card.

        • 47282847 3 days ago ago

          Sounds like you might want to dig into what these organizations do for its members besides assignment and management (not sale! you cannot own IP addresses) of shared number resources, to get a better understanding of their membership fees! I am a big fan of RIPE as an organization and appreciate their work (and less so of ARIN but I have little exposure).

          Financial reports are public, and fee structures including salaries and all work areas and work groups are decided and voted on by its members. The highest body of the RIPE non-profit is the general assembly.

          I manage two RIPE LIRs, and signup was not more work than joining any other member association. There is an annual invoice, and various payment processor options for that. I wouldn’t want it to be less “bureaucratic“ since I benefit from their processes and transparency. If they didn’t guard it, all of it would be in the hands of a Musk-like soulless broken person hiding behind a tax-evading corporate structure with zero accountability. No thank you.

          • weitendorf 3 days ago ago

            > not sale! you cannot own IP addresses

            True, but I mean, I don't own my own body either I suppose, I am just borrowing its particles from the rest of the universe. That's only a useful distinction to make if you plan on killing me.

            My personal situation is probably not very representative of most Internet users or entities interacting with the organizations that control the Internet, but I think as wireless technology improves and end-users' ability and incentive to self-host grows, they will run into the same problems that I do.

            Bottom line: I don't want to spend unreasonable amounts of time and money dealing with the idiosyncracies of the Internet Protocol and related technology, when I'm trying to do something that should be easy, like get an IP address that I can move between ISPs and cloud providers, or run an internet service from my home. It just feels incredibly wasteful to have to pay significant amounts of money to rent a number when it should be possible to claim or cheaply register one of 340,282,366,920,938,463,463,374,607,431,768,211,456 such numbers.

            Then once I nut up and pay for a small slice of the infinitely many numbers available, I have to deal with completely avoidable, godawful technical debt that only exists because the people I'm supposedly paying to govern me were so lazy that they allowed an obvious slow-motion trainwreck to play out with IPv4 over decades. They're still so lazy or cowardly or incompetent that after 20 years IPv6 availability is still only around 50%. Good thing there is an unnecessarily complicated organizational model between ICANN/IANA/RIR so that everybody can point fingers somewhere else.

            I don't want to pay for conferences and subcommittees and elaborate ceremonies for electing Vice Treasurers of RIRs, nor do I want to play tamagotchi with ranges of numbers. I just want a fucking number that allows other Internet users to connect with the stuff I put behind that number.

            I would prefer a more functional system for acquiring said numbers than one that feels all warm and fuzzy about letting the people profiting off renting numbers elect the leaders of the organizations with the authority to end rentiership of the numbers.

            • 47282847 3 days ago ago

              > it should be possible to claim or cheaply register one of

              RIPE is not the level to interact with as an end user for IP resources. LIRs act as intermediaries towards such end users. The reason why 255 IPv4 addresses is the smallest chunk you can route these days is a technical one, but apart from that IPv4s are not meant to be moved with end users. This is what DNS is for.

              As a hosting or access provider, you are meant to acquire single IP addresses or blocks from LIRs, which in turn assign and route them to a host. It is a federated, layered organizational structure.

              I get that you are upset, but I wonder who you are upset at exactly? It is not RIPEs mandate or responsibility to design Internet Protocols. If you want to argue for a better design, you should direct it at the IETF working group based on a study of the current tradeoffs, goals and technical limitations? “I want a different internet!“ Ok sure, go contribute! This openness and collaborative approach is the amazing thing about the Internet. If you have a great idea with technical merit, you will be welcomed with open arms and heard.

            • globalnode 3 days ago ago

              At least you don't live in australia, where the govt invested in a national broadband network so every aussie could have affordable and fast internet. Guess what we have. A broken cesspool of providers where its going to cost you in excess of $1K p/a to keep a connection to the internet going. Well done straya. Its the same with anything where theres the potential to fleece consumers.

          • johnecheck 3 days ago ago

            As if our only options between dysfunctional bureaucracy and corporate absolutism.

            It's not the formal processes and openness I take issue with. IPV4's ubiquity and the damage it does (funneling real money away from all of us towards ISPs) is a failure of governance.

            Though in a way it's not. It's failing me, but it wasn't designed to represent me. It's failing our species, but it wasn't designed to represent us. Who do you think it was designed to represent?

            The outcome speaks for itself.

            • weitendorf 3 days ago ago

              I for one love living in a world where ISPs, middlemen, and random internet jackpot winners were able to extract rent through a highly equitable, transparent governance model AND meet yearly at the Hilton.

        • jasonvorhe 3 days ago ago

          Just look at the origins of each of these technologies and the times in which they were created and you have all the answers you need. I'm really surprised whenever I read takes like these.

          • weitendorf 3 days ago ago

            Of course everything is a product of its time, and in 1999 or any other world where the Internet is more of a cool new thing than serious business, it makes sense. But that was 26 years ago.

            I am pretty sure the guys charging hundreds of dollars for IP addresses that cost them nothing to produce should be able to set up stripe, an identity verification product, and otherwise automate onboarding. Also, instead of writing giant process documents and slow-walking such wildly difficult problems as "allow domains to end in .cool" through infinitely nested committees they could try wielding their supreme governance over Who Owns Numbers And Names by killing off IPv4.

            As long as ICANN/IANA remain in charge of Internet governance and operate with >$100mm budgets [0] "it made sense 25 years ago" is not a valid excuse IMO.

            [0] https://www.icann.org/en/system/files/files/fy24-funding-sou...

      • eqvinox 3 days ago ago

        > Annual LIR fees at RIPE are around 2000€

        This is a recent 2025 change; we (minimum size ISP) started around 1k, it went to 1.5k in 2023 and 1.8k in 2025.

        > and includes trainings and tickets to meetings.

        Only one or two (don't remember) tickets are included with the initial becoming a member, none thereafter.

        > this seems easily doable.

        It's not negligible but not a massive expense either. Even a minimum size ISP is quickly going to be ≥1k€/month on operational expenses. Uplink is the majority for us, location rental & electricity roughly equal at ca. a quarter to third of uplink cost each.

      • throwaway63467 3 days ago ago

        It’s even less usually, they reimburse you if they have a surplus, I think I pad around 1.500 € normally.

    • max_ 3 days ago ago

      Europe is a serfdom & always has been.

      Everything is done to prop up the stature of "Lords" (the already big)

      And sqeeze out or limit the ambition of serfs wanting to reach Lordly status.

      Its a nice place if you are docile donkey that love's being taken care of by lords and have no personal agency whatsoever.

      • ta1243 3 days ago ago

        21st century Lords are the american billionaire class. They don't take care of their serfs though.

      • abc123abc123 3 days ago ago

        True. The ambitious brains flee europe, the docile remain and love it. That is why the quality of life in europe is steadily going down, and taxes steadily increasing.

        In the end, europe will be a historical museum with a tourist economy and nothing else. All industry will have moved to the US and asia.

        It's sad, but, it is also a valuable lesson for other regions on how not to destryo themselves!

        • cenamus 3 days ago ago

          All industry? Who build more cars, cranes, extreme UV chip machines, the US or the EU? Even up there in nuclear fusion

          • tmn 3 days ago ago

            Their claim was that future industry would move to Asia or us. Comparisons of US to EU for present day do nothing to contradict this claim.

            I have no take on this claim being true

  • scottgg 3 days ago ago

    This is great! I wonder how much the presenters country - the Netherlands - has made this easier with the peering. It’s hard for me to imagine just asking big serious networks to patch you in down here in Switzerland is likely to fly.

    • 47282847 3 days ago ago

      One may have a good chance talking to people behind projects like https://www.community-ix.net/ and good old in-person networking in places like RIPE meetings. Basically everyone there including people working for large telcos have a personal interest in supporting independent structures.

    • db48x 3 days ago ago

      It does seem easiest to start with an IX, where you can follow the published rules to join the IX and connection your network to it’s without bothering every other member.

      • eqvinox 3 days ago ago

        Note an IX does not give you default/full internet connectivity. You'll only be able to reach the other participants' and their affiliates' networks.

        • db48x 2 days ago ago

          True :)

          But if you’ve only ever read about BGP and this is your first time putting it into practice, an exchange seems like the easier way to gain the practice. Most of them have simple rules that you can read in advance, and often your first port is free.

        • greyface- 2 days ago ago

          Certainly not via route servers or typical bilateral peering configurations - but transit-over-exchange is a thing, if you can find an exchange participant willing to offer it to you.

          • eqvinox 2 days ago ago

            Yes, but the question is, is transit over exchange better/cheaper than buying a crossconnect? Worst case you've just introduced another dependency that costs money and adds complexity. If you're on an IXP, you're already somehow in a data centre anyway… that will have been the hard part in most cases…

            • greyface- 2 days ago ago

              > question is, is transit over exchange better/cheaper

              Better? No, almost never. Cheaper? Sometimes, especially in DCs that charge ridiculous MRCs for cross-connects. For a small/personal ASN, it can be the right choice.

    • mrngm 3 days ago ago

      Perhaps get in touch with the people behind SwiNOG? https://www.swinog.ch/ (Swiss Network Operators Group)

    • nu11r0ut3 2 days ago ago

      Hi! I am the presenter. It is true that we have a lot of active peering. In Nikhef alone there are at least 3 IXPs one can join for free (with 10G ports!). AMS-IX makes that 4 with their Bright Networks Club [1]. This can set you up with a lot of peering at basically no cost.

      If you can't get into Nikhef, you can become a member of Coloclue. One of their data centers has Frys-IX indoors, and members can get an XC there.

      [1] https://events.ams-ix.net/bright-networks-club

    • preisschild 3 days ago ago

      I thought this would be the easiest in Switzerland with homelab-friendly ISPs like Init7

    • eqvinox 3 days ago ago

      Uh, Switzerland is one of the best locations to do this in, due to the government fibre build-out mandate. However, you have to start a little larger than "minimum" ISP size in other countries to be able to take advantage of it.

      Peering… really depends on where you are, it hasn't been a problem for us.

      That said, Init7 is, for the time being, still a scaled-up mini ISP. They're slowly devolving into corporate-dom, but not there yet.

      • theideaofcoffee 2 days ago ago

        I’m interested to hear more about Init7 becoming more corporate, is that just by virtue of growth and having to adapt to the complexity of being a larger org? Or something else?

        • eqvinox 2 days ago ago

          It's just market pressure to make/keep things cheap (= "optimising" support), and also with becoming larger it's harder to reach the 'tech nerds' as normal customer if you don't have another social connection to them.

  • Arelius 2 days ago ago

    Been looking into this for a while myself, very cool. It seems .nl has a lot of resources and support for this sort of thing. Any similar resources like coloclue in the San Francisco Bay Area? It seems confusing to find just a couple of U of rack space in SF that you could peer in and out of?