[HN Gopher] A PCIe network interface card that adds full router ...
___________________________________________________________________
A PCIe network interface card that adds full router capabilities to
your servers
Author : Alupis
Score : 157 points
Date : 2022-03-28 16:18 UTC (6 hours ago)
(HTM) web link (mikrotik.com)
(TXT) w3m dump (mikrotik.com)
| johnklos wrote:
| Proprietary OS from a company that has gotten caught with their
| proverbial security pants down around the ankles? No, thank you.
|
| When this can run non-Mikrotik open source software, this'll be
| great!
| pilsetnieks wrote:
| > that has gotten caught with their proverbial security pants
| down around the ankles
|
| So just like any other major networking provider, including
| opensource projects?
| oliwarner wrote:
| An embedded device running inside my server I cannot audit, with
| direct memory access to everything running?
|
| Feels like the Holy Grail of backdoors.
| cduzz wrote:
| A modern (server) system probably has 3-8 of these already,
| some of them explicitly with independent network connectivity.
|
| Trust your vendors, lock down your network, be large enough to
| build your stack yourself; chose any 2...
| blibble wrote:
| turn on your IOMMU
| icedchai wrote:
| Many servers already have embedded devices in them that you
| can't audit. How is this anything new? (I'm thinking of remote
| management, like HP iLO, Intel AMT, etc.)
| runnerup wrote:
| And secret microcode / hidden instructions in every major x86
| CPU, presumably for the NSA.
|
| [0]: https://www.youtube.com/watch?v=KrksBdWcZgQ
| ACAVJW4H wrote:
| STH has a bit more information on the card
|
| https://www.servethehome.com/mikrotik-ccr2004-1g-2xs-pcie-is...
| kkielhofner wrote:
| Neat concept but I wonder why the PCIe initialization delay can't
| be handled with an option ROM. I don't know that a fully fledged
| option ROM would add value but it seems like it could be a good
| workaround/hack to not require additional BIOS configuration or
| support a BIOS that doesn't allow configuration of a delay.
|
| I've seen some option ROMs take 10 seconds or more depending on
| the card - hardware RAID controllers being a well known example.
| wmf wrote:
| Mikrotik probably can't afford to develop an option ROM.
| pilsetnieks wrote:
| What makes an option ROM so expensive?
| rsync wrote:
| Interesting ... so if I could find a server board with _no other
| network ports_ and then put this card in, I could _finally_ build
| a wire-speed multi-gigabit "network slug"[1].
|
| [1] https://john.kozubik.com/pub/NetworkSlug/tip.html
| bombcar wrote:
| In theory you could configure RouterOS to be your slug itself,
| and provide PCIe power and _no computer at all_ to slug this.
| runnerup wrote:
| Just watch out for Amazon Sidewalk! Your consumer TV could
| connect to your neighbors' Amazon Echo wirelessly to continue
| sending screenshots (or hashes of screenshots) to Amazon and
| its marketing partners.
|
| [0]:
| https://www.washingtonpost.com/technology/2021/06/07/amazon-...
| candiddevmike wrote:
| My Linux server already has "full router capabilities" AND I
| don't have to use RouterOS to configure it (which is just a shit
| abstraction on top of common Linux network services like
| iptables).
| walrus01 wrote:
| Almost everyone I know that's ever used JunOS from a command
| line for 'serious' ISP things finds RouterOS painful and
| cumbersome.
|
| The way things are laid out in a hierarchy in a full system
| "/export" from a Mikrotik is so weird and annoying compared to
| a hierarchical junos configuration from a "show configuration"
| on a juniper router.
|
| If people want to make a real router of an x86-64 system rather
| than putting a mikrotik pci-e card into it (wtf, why?) I'd
| recommend they go with vyatta or VyOS instead, or install
| something like a barebones centos or debian and then add FRR to
| it.
| barbazoo wrote:
| > putting a mikrotik pci-e card into it (wtf, why?)
|
| It's in the first sentence of the post:
|
| > Save space in your server room
| walrus01 wrote:
| if you want a mikrotik, buy a mikrotik hardware 1U router,
| despite the many issues with them the one thing they do
| have going for them are low power consumption and small
| space use. an actual ccr2004 1U box is not that large and
| can be mounted almost anywhere.
| tremon wrote:
| Maybe I'm dense, but wouldn't that solution still use 1U
| more space than the PCIe card mentioned in TFA?
| vetinari wrote:
| In your own rack, you would do exactly that. But if you
| paying per U in colo, this card can save you one slot.
| walrus01 wrote:
| If you have enough traffic to need multiple SFP28
| interfaces in colo and can't pay $150-250/mo extra to put
| in place a real hardware router, or stop paying by the 1U
| increment and get 1/4, 1/3 or 1/2 of of a cabinet,
| priorities and risk tolerance are misaligned in my
| opinion.
|
| if you have >10Gbps traffic flows and are putting the
| router and other hosting environment/linux things all
| together in one 1U piece of hardware as a single x86-64
| server, that's a "too many eggs in one basket" problem.
|
| also worth noting that many colo/hosting ISPs won't offer
| 25GbE circuits on SFP28 anyways, you can buy either a
| 10GbE transit link or 100GbE, or maybe 2x10GbE bundled
| together in a 802.3ad or similar.
| vetinari wrote:
| In this case, I was thinking about moving a currently
| half a rack worth of equipment from premises to colo, as
| the (internal) users are mostly on WFH anyway. They would
| not generate 1 Gbps of external traffic, not even in
| spikes. Currently, as it is, it makes more sense to stay
| on premises, but if some increase of density happened, it
| could make some sense.
|
| However, it is not going to happen, it would be somewhere
| at bottom with priority. It was just an exercise, what
| could be done.
| oarsinsync wrote:
| As a network engineer who's worked on Cisco, Juniper,
| Foundry, Brocade, Extreme, HP, Dell, and even Netgear, let me
| assure you that while the urban legend is that "JunOS is IOS
| done right", the reality is that they're all terrible in
| their own ways.
|
| JunOS is generally better than IOS(-XR), but it's still got
| its sharp edges. VyOS / Vyatta are poor enough clones that
| they will bite and _seriously_ suck to anyone who's actually
| got real JunOS experience.
|
| Let's be real. The goal in improving network configuration
| standards is to _suck less_. That's it. Everything in
| networks sucks. Anyone who tells you otherwise either lacks
| experience in general, lacks experience suffering at the
| bleeding edge, or lacks my cynicism and genuinely sees the
| world as a better place than I do (I envy them for any of the
| above)
| walrus01 wrote:
| I don't disagree with any of this - have been using JunOS
| since the M40 was the absolute apex of service provider
| core router technology. Lots and lots of weird bugs in
| various versions of IOS and JunOS on all their platorms.
|
| Big difference between what you might get spending $15,000
| for a Juniper MX204 running JunOS and a Mikrotik $800
| router. I mentally categorize Mikrotik RouterOS and similar
| ultra low cost things in the same tier as VyOS. It's
| _cheap_ but there are tradeoffs to going cheap. One has to
| understand the risks and tradeoffs of running a lot of your
| traffic or important things through cheap routers.
| Sometimes it 's a risk worth taking.
|
| Foundry, as we've seen, was a straight knockoff of the IOS
| 12.2/12.4 CLI and interface. Used plenty of Foundry
| switches in a previous role.
|
| Everything does suck. Some things suck less. Sometimes you
| can pay money to get things that suck less.
| oarsinsync wrote:
| > Everything does suck. Some things suck less. Sometimes
| you can pay money to get things that suck less.
|
| And sometimes you pay more money and you're the one being
| made to do the sucking :-\
| iso1210 wrote:
| > Everything does suck. Some things suck less. Sometimes
| you can pay money to get things that suck less.
|
| And then there's Cisco
| lormayna wrote:
| I have worked for a medium size ISP and we had Juniper,
| Cisco and lot of Mikrotiks. For me the big lack in
| Mikrotik, compared to the bigger vendor, is the lack of
| real support. No TAC services, no SLA, etc. The only way
| to get support is via email, but you have to wait days
| for a response. And also the system is not stable like
| the one from big vendors. Anyway, the performances of
| Mikrotik are impressive for the cost.
| kazen44 wrote:
| and TAC/support is half the reason you buy from the known
| vendors in the first place. (the other being well rounded
| and actual trustworthy performance numbers when using
| more niche network technologies, especially in regards to
| encapsulation).
|
| for a comparison, I once had an issue where both routers
| in a redundant setup failed within half an hour of each
| other. (was a pure coincidence, the setup was redundant).
| then, the sparefallback unit would not boot, and jtac
| send us a replacement within 3 HOURS...
| techsupporter wrote:
| > make a real router of an x86-64 system rather than putting
| a mikrotik pci-e card into it (wtf, why?) I'd recommend they
| go with vyatta or VyOS instead
|
| One thing I've been looking for is a hardware box that can
| replicate what Ubiquiti's EdgeRouter Infinity does: a handful
| of 10Gbps SFP+ ports (sorry, I know that the term is "cages"
| but I just can't) and a couple of copper 1Gbps ports.
|
| So far I haven't found anything but I feel like my search
| will get motivated in the next couple of years since it feels
| like Ubiquiti has forgotten that EdgeRouter exists.
|
| Do you have any rack form factor x86-type systems you like
| for VyOS?
| walrus01 wrote:
| When space permits I prefer full-size 1U systems that have
| dual/hotswap power supplies and room for three low profile
| pci-e slots, such as a Dell R630/R640 or similar. With
| Intel chipset 4-port 10GbE SFP+ NICs this would max out at
| twelve ports plus whatever is on the motherboard
| daughtercard for network interfaces (2 x 10GbE + 2 x 1GbE
| copper, or whatever).
|
| for smaller or shallow stuff, supermicro, msi, tyan, asus
| logifail wrote:
| > a hardware box [with] a handful of 10Gbps SFP+ [..] and a
| couple of copper 1Gbps ports
|
| I have a couple of (fanless!) CRS305-1G-4S+IN[0] at home,
| one in my study and one in the utility room. They each
| connect with 10GbE fibre (or DAC) to ConnectX-3 cards in my
| PCs and servers.
|
| [0] https://mikrotik.com/product/crs305_1g_4s_in
| techsupporter wrote:
| I appreciate the recommendation but that's kind of a gap
| from the EdgeRouter Infinity (ER-8-XG). The Infinity has
| 8x10Gbps SFP+ ports, a single copper 1Gbps port, 16GB of
| RAM, and a multi-core processor because it's designed as
| an inexpensive core router for a mid-sized network.
|
| Where I work, we use one of them as our main router with
| multiple peering sessions and two transit uplinks.
| According to Cacti, right now we're pushing about 30Gbps
| through the router.
|
| That's what I'm looking to eventually replace, if
| Ubiquiti doesn't start up with software updates to the
| EdgeRouter line again. But I think that's the problem:
| the EdgeRouter line is so amazingly inexpensive for all
| of the power you get, there's no financial incentive for
| Ubiquiti to invest in it and all of the players with the
| "proper" routers--the Junipers and Ciscos and the like--
| start at three times the price of an ER-8-XG.
| logifail wrote:
| > that's kind of a gap from the EdgeRouter Infinity
| (ER-8-XG)
|
| Indeed, not least on price. How much was your ER-8-XG? My
| CRS305-1G-4S+IN were about USD180 each.
|
| EDIT: If there were a silent version of the
| CRS326-24S+2Q+RM[0][1] I'd have bought one already...
|
| "The MikroTik CRS326-24S+2Q+RM is an insane switch. Its
| specs are relatively mundane by modern standards. It has
| 24x SFP+ 10GbE ports and 2x QSFP+ 40GbE ports making it
| not even as powerful as mainstream previous-generation
| switches like the QCT QuantaMesh T3048-LY8 that we
| installed in our lab years ago. Instead what makes the
| switch insane is that it offers all of that performance
| at $475"
|
| [0] https://mikrotik.com/product/crs326_24s_2q_rm [1]
| https://www.servethehome.com/mikrotik-crs326-24s2qrm-
| review-...
| walrus01 wrote:
| a crs326 is a layer 2 switch - not comparable with a
| router. you could categorize it as more like a cisco
| 3750G from ten years ago in capability of 24 ports of
| copper gigabit in one place.
|
| any mikrotik CRS series has very limited routing/layer 3
| ability compared to a CCR series. Different things for
| different purposes.
|
| look at the logical block diagrams mikrotik provides of
| their crs series equipment. it's all a bunch of ethernet
| switch chips in a few blocks of 8 ports and then
| something like a single 1GbE link to the CPU. the moment
| you start telling it to do layer 3 things its capability
| is very limited.
|
| https://i.mt.lv/cdn/product_files/CRS326_180248.png
| Alupis wrote:
| For what it's worth - there is a healthy "modding"
| community for some of these Mikrotik switches. People
| convert them into fanless/silent units pretty regularly,
| or swap the fans for higher flow / lower rpm fans, etc.
| vetinari wrote:
| Have look at Mikrotik CCR2004-1G-12S+2XS (1G-12S+2XS
| means 1x1Gbps RJ45, 12xSFP+, 2xSFP28) or CCR2116-12G-4S+
| (12G-4S+ = 12x1Gbps RJ45, 4xSFP+), depending how many
| ports and what kind of routing performance you need
| (check the block diagrams, they tell the story).
|
| However, neither of them will route 80 Gbps full duplex.
|
| Then there is CCR2216-1G-12XS-2XQ (1x1Gbps, 12xSFP28,
| 2xQSPF28); this one is supposedly capable of routing shy
| of 200 Gbps @1518 packet size.
|
| Edit: another thing on Mikrotik naming conventions: CRS =
| switches; CCR = routers.
| walrus01 wrote:
| If people have anywhere _near_ 80 to 200 Gbps of real
| world IP traffic and are thinking of using a mikrotik for
| it, they seriously need to re-examine the revenue from
| customers that 's going through that >50Gbps of traffic,
| business risk profile and how serious they are about
| things...
|
| At that scale you'd better have a redundant identical
| twin pair of routers with 1+1 or N+1 redundant everything
| (fans, power supplies, routing engines, etc) 24x7x365
| service contract, and so on. Not something you can or
| should do with mikrotik.
| kazen44 wrote:
| juniper mx204 would be a great box for this..
|
| but far pricier then mikrotik..
| gonzo wrote:
| 10gbps at full-size packets is 812,743pps
| 10,000,000,000/(1538*8) = 812,743.82
|
| 200gbps is 20x this rate, or 16,254,876pps
|
| This is 9% higher than the 10gbps packet rate for 'line
| rate', 14,880,952 pps, which can be done on a single core
| these days.
|
| https://docs.fd.io/csit/rls1807/report/detailed_test_resu
| lts...
| vetinari wrote:
| They do indeed claim 16 254,8 kpps. They have l3hw
| offload - so not every packet needs to go via cpu - and
| 16 cores.
| techsupporter wrote:
| > Have look at Mikrotik CCR2004-1G-12S+2XS (1G-12S+2XS
| means 1x1Gbps RJ45, 12xSFP+, 2xSFP28) or CCR2116-12G-4S+
|
| Both of these look fantastic. The second one, with the
| four SFP+ ports, looks like an almost drop-in replacement
| for the Infinity, particularly with its 16GB of RAM. (We
| use soft-reconfiguration inbound which bloats the amount
| of RAM needed for the tables.)
|
| > However, neither of them will route 80 Gbps full
| duplex.
|
| That's actually fine, at least for our needs. We only
| have 50Gbps of connectivity between peer, IXP, and
| transit links and today's 30Gbps is high because of end-
| of-month activities. We got the Infinity largely because
| it was the _only_ EdgeRouter that could do what we
| needed. Like the gap between EdgeRouter Infinity and
| "every other router that can do what it does," there's a
| rather large gap in Ubiquiti's EdgeRouter line. The next
| one down in the list is the EdgeRouter-12 that is a small
| fraction of the capability of the Infinity.
|
| > another thing on Mikrotik naming conventions: CRS =
| switches; CCR = routers
|
| That's good to know. I hadn't started down the Mikrotik
| path yet but I'll give it a look. We have a leaf router
| at a small office where we experiment and maybe I can put
| one in there to start.
|
| Thanks for all of the information!
| stingraycharles wrote:
| As someone who's a home networking enthusiast, and has too
| much Mikrotik gear at home, I can kind of understand what
| they're coming from. RouterOS has the usability of
| "enterprise-grade" network equipment (meaning it's arcane and
| non-intuitive), but at the same time has lots and lots of
| half-working features.
|
| I simply cannot believe how terrible their IPv6 support is
| (still no connection tracking!), and plenty of weird
| glitches, etc.
|
| But! Their hardware is very reasonably priced, and an
| excellent gateway to "real" networking equipment for the
| hobbyist. It's unfair to compare it against Juniper and the
| likes: yes, it's much better, but yes, the products are also
| 10x - 100x as expensive.
|
| While everything that's done in RouterOS can also be done
| under vanilla Linux, I buy Mikrotik precisely because I don't
| want to build a custom Linux router. I want something that
| comes with a GUI, and I won't have to spend too much time
| setting up.
|
| Having said that, I would absolutely kill for an "escape"
| Linux shell. I _know_ that RED supports ECN in Linux, please
| allow me to use it!
| blibble wrote:
| > I simply cannot believe how terrible their IPv6 support
| is (still no connection tracking!)
|
| I see a list of connections under "IPv6 firewall" under the
| connections tab?
|
| > and plenty of weird glitches
|
| this bit however I agree with
| stingraycharles wrote:
| I don't think that contrack based mangle rules work,
| though. If it does, it must be a recent fix (I'm on
| ROS7.1)
| iso1210 wrote:
| 7.x is still effectively in beta, there are many features
| that don't work yet, last time I checked neither
| multicast nor bfd were working.
| walrus01 wrote:
| the idea that somebody thought to ship to production
| release a router operating system with broken bfd is
| amazing.
| Alupis wrote:
| Well, they didn't - not really at least.
|
| 7.1 is only required on their brand new router targeted
| at enthusiast home users. The RB5009, which specifically
| says it's targeting home labs and explicitly came with
| the caveat of 7.1 being the minimum version and there is
| no LTS in the 7.x branch as-of yet. This is the only
| product that requires the 7.x branch.
|
| Everything else ships with 6.48.x LTS or 6.49.x Stable.
| Nearly all serious users are using the LTS branch. The
| 7.x branch is well known within the RouterOS community to
| not be "production" ready... although that's where new
| features and stuff are going. It will be, one day.
| sleepydog wrote:
| > still no connection tracking!
|
| Seriously? Is it not possible to have stateful firewall
| rules for IPv6 traffic? Or is it just NAT that won't work
| (I don't care about NAT, NAT can die)? I was considering
| getting a microtik router but this would be a dealbreaker.
| flower-giraffe wrote:
| SRP of 199usd and 2x SFP28 25GbE.
|
| It's not for the enterprise but I'll get some for home.
| [deleted]
| [deleted]
| cute_boi wrote:
| I think the use case is to reduce CPU usage. Its like GPU
| cards, but for networking.
| xxs wrote:
| By just reading the title you can tell it runs an Arm cpu
| with linux on it. Not really certain how useful that is.
| 293984j29384 wrote:
| I'm not sure where the confusion is. OP mentioned that his
| Linux system can already do routing. The purpose of this
| card is to remove that load from the computer. The
| manufacture suggests it can do up to 100Gbps which isn't
| trivial.
| drewg123 wrote:
| _This NIC can reach wire-speed (100Gbps) with Jumbo
| frames._
|
| To me, this suggests that it's packet-rate limited, and
| if so, it can really only be counted on to do 1500/9000
| or ~16.6Gb/s with standard frames.
| xxs wrote:
| >>reduce CPU usage. Its like GPU cards,
|
| It uses another CPU to do that. GPU is fundamentally
| different, high memory bandwidth, embarrassingly
| parallel, virtually no branches, and what not. That's
| just using a different CPU to do more CPU, and using the
| same OS the host already runs.
|
| Then it requires its own security maintenance (+training)
| and patches.
| jotm wrote:
| I think the analogy was that a CPU can do a GPU's job,
| but a GPU will do it much faster.
|
| Pretty much all modern NICs are already using separate
| hardware to reduce the load on the main CPU. I.e. using a
| different CPU to do more CPU.
|
| Without that you're looking at sacrificing a whole core
| or two just to handle 1Gbps, nevermind 10+.
| benou wrote:
| Personally I think one of the real usecase for smartnic
| is isolation: for a cloud provider, you can rent a bare
| metal instance and run all your networking security stack
| (think encapsulation, filtering, throttling etc) on the
| smartnic.
|
| IOW the customer has full control of the host, but the
| cloud provider manages the smartnic. Incidentally, this
| is exactly what AWS does with their ENA adapters designed
| by... (ex-?)Anapurna Lab they bought some years ago (:
| jabart wrote:
| Mikrotik uses Annapurna ARM chips.
| aseipp wrote:
| Products like this are, generally speaking, designed for
| service providers, where having more available host
| capacity directly translates to increased revenue.
|
| Consider a cloud provider who offers virtual machines to
| users: the physical host machine typically is involved in
| whatever networking path is necessary (e.g. an SDN), as
| well as the control plane software for managing VMs, and
| other tidbits. Moving the entire networking and SDN layer
| off the host system and onto an accelerator card, with your
| own customizations to the data path, means you can take
| those host resources and use them for VMs instead --
| effectively increasing the total amount of capacity you
| have available. It's not just CPU time either: things like
| this also effectively increase available PCIe bandwidth,
| memory bandwidth, etc, available to users, by moving the
| resources the operator needs elsewhere.
|
| There are some other benefits too, like you can run the
| whole security framework on a card like this. Or QoS
| controls. You could for example rent out the entire bare
| metal server to someone more or less and use a device like
| this to implement throttling/QoS/SDN transparently.
|
| Most of the vendors are calling these "Data Processing
| Units" or "Infrastructure Processing Units" or whatever,
| but the idea is all the same. Offloading the
| networking/data paths into accelerators allows you to offer
| more raw compute to your users. For example, Nvidia
| Bluefield or Intel's new Mount Evans IPU.
|
| This Mikrotik is basically the bargain-bin version of those
| products. Which is actually pretty cool. I could actually
| use a couple of 25GbE breakouts for that price...
| Alupis wrote:
| This Dual SFP28 (dual 25Gb cages) plus 1Gb Eth PCI-e card has
| an MSRP of $199, meaning a street price will be a bit under
| that.
|
| 10Gb NIC's run around $100... and can't do any switching or
| routing. As mentioned, this card can offload 100% of routing
| needs from the server (ie. zero CPU usage on your server to
| make routing decisions), can switch at line speed (well above
| line speed actually, rated for 100Gbps throughput), plus the
| server can still use one of the ports for it's own needs.
| Sounds pretty powerful to me.
|
| It's unlikely this is an interesting product for a home lab or
| business - it's likely more geared towards service providers.
| Still a pretty cool idea none-the-less, regardless of how you
| feel about routerOS.
| cute_boi wrote:
| 10/100/1000 Ethernet ports
|
| I wonder why they need to support 10 mbps port? Is it just
| because if the card supports 1000 mbps it will support 10mbps
| effortlessly?
| jacquesm wrote:
| You get the 10 mbps capability for free because that is what
| the auto-negotiation protocol will use:
|
| https://en.wikipedia.org/wiki/Autonegotiation#Electrical_sig...
| xxs wrote:
| yes, 10 is just a single twisted pair and if the cable is
| faulty the connection can degrade to it.
| jaywalk wrote:
| I have had bad cables degrade from 1000 to 100, and one time
| had to force a shoddy (and very temporary) connection to 10
| for it to work at all. So there is definitely a use for it.
| Maxburn wrote:
| I still have quite a lot of equipment in the field that is
| 10/half. PLC's that control commercial HVAC are expected to
| last the life of the building, at least until a refurb or
| two.
|
| Cisco has some switches that can't go down to 10, which
| makes it interesting when those show up on site and the
| HVAC system can't link up any more.
| rubatuga wrote:
| Nope, you need two twisted pairs!
| hnlmorg wrote:
| 10baseT is a single twisted pair. It's 100baseT that
| requires two twisted pairs but that's 100Mbs rather than
| 10Mbs.
|
| It used to be common run 10Mbs over coax too, back before
| Ethernet took over.
| assttoasstmgr wrote:
| This is simply incorrect. 10Base-T is two pairs, one TX
| one RX. Source: am expert, have designed low level
| ethernet hardware.
|
| It amazes me how much misinformation gets posted on HN
| with convincing authority.
| hnlmorg wrote:
| 10base-T1S and 10base-T1L are single pair. Though I
| didn't realise they're a modern standard until I just
| looked it up.
|
| Coax is also two "wires", though obviously not twisted.
|
| I used to do networking professionally too. Though it
| looks like I've gotten rather rusty on the basics.
| Dylan16807 wrote:
| Citation needed.
|
| There's a 10BASE-T1 but this says it's very recent?
|
| 10BASE-T and 100BASE-TX are very similar except for the
| line encoding. One pair each way.
|
| Coax uses one line, but that's not using twisted pairs at
| all.
| assttoasstmgr wrote:
| *-T1 Ethernet was designed by Broadcom and the car
| manufacturers to implement single pair ethernet for
| automotive applications. Specifically for things like
| backup cameras, ADAS, etc. The standard is less than 10
| years old and has nothing to do with 10base-T.
|
| https://en.wikipedia.org/wiki/BroadR-Reach
|
| 100Base-T1 has more in common with 1000BASE-T than the
| legacy standards, imagine if you took a single pair from
| the 4 needed to do Gigabit.
| Dylan16807 wrote:
| And it has to be a particular two. It's a very situational
| bonus rather than proper graceful degradation.
| cft wrote:
| They advertise it as having "full routing capabilities", but I am
| not sure if 4GB of RAM can keep the full ipv4/6 routing tables?
|
| Edit: never mind, it says it's primarily for home use
| Alupis wrote:
| I'm not certain what you mean. 4GB of RAM is far more than
| plenty for nearly anything. This card isn't going to be the
| core router for Comcast or anything... but for what it's worth
| BGP definitely requires far less than 4GB of RAM, although it
| depends on the exact implementation of course.
|
| And this card is highly unlikely to be targeted for home use -
| mostly service providers doing routing within their private
| networks.
| yewenjie wrote:
| In layperson terms, what are these 'full router capabilities' and
| why would one want those?
| synergy20 wrote:
| Knowing Mikrotik for like 2 decades, it should do better than
| UBNT really. Mikrotik still produces great hardware, but it's
| totally eclipsed by Ubiquitous Networks these years. It's kind of
| like watching digitalocean the new cool kid playing the same
| tricks overtakes linode, sigh.
| core-utility wrote:
| Mikrotik misses the "polished" aspect still, that UBNT does
| well. As someone with moderate enterprise network experience,
| setting up RouterOS as a basic L3 switch was way more difficult
| than it should have been. That being said, once I was done I
| haven't had to think twice about the switch, it just works
| (which should be default, but isn't always the case).
| Alupis wrote:
| Probably different target audiences. Mikrotik originally got
| big with WISP's years back, where it was common to have
| Mikrotik handling routing and UBNT handling wireless
| PtP/PtMP.
|
| I've found UBNT's modern switches and routers to be nice from
| a UI perspective - but oh boy do they have strong opinions on
| how you should configure them. You have to jump through a ton
| of hoops to get the Dream Machine Pro to _not_ be your actual
| gateway, for instance... tricking it into thinking it 's the
| gateway and then unplugging that port, etc.
|
| Mikrotik is happy to let you do whatever you want, to your
| detriment sometimes.
|
| UBNT gear seems great for SMB/Home Labs where people just
| want it to work... Mikrotik is for those who want to tinker,
| and more power-oriented users looking for non-conventional
| setups.
| vetinari wrote:
| I have mixed experiences with UBNT polish. It looks good on
| screenshots, it allows to set up simple things, but there it
| ends. It is often inpractical, shows nonsense data (basically
| anything dashboard is just random, useless data with zero
| relevance) and if you want something slightly unexpected
| (like ipsec tunnels defined by hostnames and not by ip
| addresses), you are either stuck with json (on older models
| with config.gateway.json) or it is straight impossible.
|
| RouterOS did have a learning curve, and there are some
| unexpected bugs, but compared to UBNT, I like it much more.
| Yes, it has more knobs, and they generally allow configuring
| that needs to be done.
| gh02t wrote:
| To be fair to Mikrotik if you just want basic/intermediate
| switch they have SwOS, which is FAR easier to set up. I also
| find RouterOS to be extremely unituitive, but SwOS is a
| breeze. I think most of their switches can run either and
| even dual boot.
| synergy20 wrote:
| what's the goal for SwOS(new to me), replacing RouterOS?
| vetinari wrote:
| No, SwOS is a simple OS only for switches; it's purpose
| is to configure the switch chip and then get out of the
| way.
|
| I do not like it, it is configurable only via web. No
| cli, no api, no ansible/terraform-like automation
| possible.
| AdrianB1 wrote:
| The first time when I read about it on ServeTheHome I had no idea
| what this can be used for. Then I saw the price and my jaw
| dropped, it is cheaper than a basic NIC with dual 25 Gbps ports.
| Together with the CPU and RAM on it, it makes a lot of sense for
| specific use cases and the price is appealing: for a Small or
| Medium Business with some servers and not a lot of dedicated
| network equipment, it allows to move the router/firewall inside
| the server case, combining it with the NIC at a good price and
| without eating up any of the server resources.
|
| Do you want a cheap dual-port NIC at 25Gbps? How about we add
| some solid router capabilities on it for no extra price?
| compsciphd wrote:
| used Mellanox cx3 (qfsp, 40gbps) cards go for $30 or so on ebay
| and can go lower (I bought 5 a while back for $75 total).
| vetinari wrote:
| How long ago was that? I've bought recently newish dual-port
| (SFP+, 10Gbit) Connect-X 3 Pro at 80 GBP per piece. And that
| was one of the better prices.
| kube-system wrote:
| The older QSFP cards go for very cheap here in the US. $20
| right now on eBay. SFPs go for more, and even more for the
| dual interface cards.
| compsciphd wrote:
| this is the cheapest I see right now on ebay for dual port
| card ($35) so perhaps a bit higher than what I remember
| from a year ago (I guess silicon shortage effects
| everything).
|
| https://www.ebay.com/itm/265592690915
| nanochad wrote:
| Routing should be done in software.
| vetinari wrote:
| It is.
|
| Just the software doing the routing is not running on your main
| CPU, but on the CPU bundled on the board.
| kazen44 wrote:
| please define routing?
|
| actual packet forwarding should be done in hardware, because
| software forwarding has atrocious performance in comparison.
| egberts1 wrote:
| Not seeing any mention of Data Center Bridging Protocol there.
|
| https://en.m.wikipedia.org/wiki/Data_center_bridging
| wmf wrote:
| Yeah, at that price it may be missing some features you'd
| expect from a normal NIC.
| oneplane wrote:
| Essentially it's a single board computer with two network
| interfaces, one on the PCIe side, one on the bracket side.
|
| This has been done before with the likes of DSL modems that
| weren't actually modems but just router-on-a-card that would just
| have a Realtek PCI chip on the bus side, which then directly had
| its GMII interface hooked up to a conexant DSL modem/router
| package which itself then connected to the actual on-board modem.
| zelon88 wrote:
| So can you add more regular NICs and then use them as router
| ports?
| Nextgrid wrote:
| No. The ports on this "NIC" are actually connected to the
| router, though they can be passed through to the host if
| needed.
|
| The ports on another NIC would be assigned directly to the
| host. While I'm sure you can theoretically redirect them to
| this router wit a combination of VLANs and other Linux
| networking magic, you will be limited by your CPU and it's
| unlikely you'll manage more than a few Gbps.
| Melatonic wrote:
| While this seems cool for some implementations there is a reason
| we often have separate boxes for compute / storage / routing.
| Some of these are much more critical to have consistently running
| than the others and it also means it is easier to swap out and do
| upgrades without having to worry about affecting the other parts
| of the pie. I think virtualized networking devices like routers
| are definitely the future but I would still much rather have it
| as its own separate physical box so that if some hardware fault
| in a server takes it down the network still functions (not to
| mention having them on different UPS hardware or different levels
| of redundancy.) And with servers getting smaller and smaller and
| the compute required getting more and more power friendly I do
| not see this as something I would like to use unless I was
| EXTREMELY space constrained.
|
| Where I can see this being super cool though is niche use cases
| like highly portable servers and whatnot for things like VFX
| shoots. I once was contracted to built a set of highly mobile and
| durable servers for mobile rendering of 8K footage. I built the
| servers into some super durable hard case boxes that are usually
| used for shipping things like expensive camera equipment,
| military hardware, etc. The cases even have a valve to equalize
| pressure in case they get pushed deep underwater (like in the
| event of a boat capsizing) and a very robust waterproof gasket.
| Of course for the servers to be running the case must be open
| (mainly for cooling) but it would have been interesting to
| network multiple of them together AND other equipment without
| needing a separate physical device for routing. It would also
| have made scaling the system much easier if each server could
| also act as a router - you could bring one or 10 and each could
| function independently of each other.
| SamuelAdams wrote:
| I currently have a ProtectCLI vault device running PFSense for my
| router. I also have a TrueNas / FreeNas device (Supermicro board
| with Xeon 26xx processor, 2x 1Gbps ports).
|
| I've been wanting 10 Gbps networking for some time but I've been
| undecided how to best do that. Could I simply get this card, drop
| it in my FreeNas box, then plug my Arris S33 modem into the card,
| then the card to my network switch? Would the FreeNas host also
| get 10/25 Gbps virtually, or do I still need another card
| specifically for the FreeNas box?
| mjh2539 wrote:
| Your switches and all client devices would have to have 10Gbps+
| NICs/be 10Gbps+ capable.
| bombcar wrote:
| I got the four SPF+ port microtic, some eBay 10GB cards for my
| VM server and my ZFS NAS, and connected one port to each, along
| with one to the 10GB uplink on the old Nortel switch and one to
| the 10GB port on the Mac (that one is the only one that was
| cable ethernet instead of fibre or direct connect).
|
| Works fast and well. The fifth "management" 1GB port goes to
| my router, 1GB is way faster than my internet anyway.
| nimbius wrote:
| im seeing a lot of "my router" and "my computer" threads so its
| probably worth it to say this isnt for your home network.
| Mikrotik is targeting larger customers with a product that
| handles offloading to the ASIC's on the board, which is far more
| performant and scalable than COTS ethernet cards or the onboard
| gigabit.
|
| the reason you would slap a router card in your rackmount server
| is because its an IOMMU passthrough to a k8s service load
| balancer or straight up just openstack and the push toward
| hyperconvergence. the switch is already virtual inside the kvm on
| openvswitch (has been for a decade now), but the router is still
| hardware and this product aims to solve that problem.
| aseipp wrote:
| You aren't wrong but honestly I'm having a hard-time
| envisioning a target audience for this device _besides_ the
| ardent homelab crowd, or existing Microtik users who just want
| to eliminate one more piece of gear like a normal CCR from
| their setup and move it into the server itself. I don 't see
| many "larger customers" moving to something like this instead
| of competitors. It's not like it's priced out of homelabs; $200
| MSRP is the price of an entry level 2x10G Intel card and I'd
| consider that table stakes for actually adventurous home
| networking.
|
| The bandwidth on the interfaces isn't high enough to match most
| enterprise customers needs -- 25GBe/40GBe had pretty marginal
| market penetration compared to 10G where you don't need
| hyperconverged solutions, and beyond that most major
| hyperscalers and others have skipped straight to 100G as far as
| I can see, to leverage economies of scale. And the CPU complex
| and ASIC together aren't powerful enough with enough resources
| to offload serious "service provider compute" workloads to;
| they even note specifically things like it reaches "line rate
| with Jumbo Frames", where most of those other solutions aim for
| line rate @ MTU, so I'm suspicious of that wording. And on top
| of that you need some actual dedicated engineering (operations,
| engineers) to utilize a solution like this versus just
| reserving AWS instances with ENA adapters or whatever. Anything
| this can do, something like Bluefield will just do better in
| every way, if you need the hardware yourself.
|
| So I legitimately have a hard time envisioning anyone other
| than random nerds buying these. Any large customer is probably
| better off just going with Nvidia (Bluefield) or Intel (Mount
| Evans). But hey, for two 25GBe ports at the price of a normal
| 10GBe card, as long as I can pass them through directly I
| suppose I can handle RouterOS or whatever, and if the software
| gets more advanced that's cool too. And if it gets more people
| on the whole converged infrastructure bandwagon, sounds good!
| antattack wrote:
| This card could be good if one is leasing rack space so the
| router now takes up the same space as the server.
| walrus01 wrote:
| > honestly I'm having a hard-time envisioning a target
| audience for this device besides the ardent homelab crowd,
|
| 1. people running weird janky WISPs, like, two guys and a
| pickup truck in some very rural parts of the USA. usually
| very budget limited.
|
| 2. small very budget limited ISPs in the developing world.
|
| everyone else in the service provider is not using a $200
| mikrotik to do serious routing of >10Gbps of traffic.
| mechanical_bear wrote:
| > people running weird janky WISPs, like, two guys and a
| pickup truck in some very rural parts of the USA.
|
| I may have been involved with those guys at some point...
| depereo wrote:
| 25 to the server is pretty popular in mid-tier IaaS
| providers. Means you can use 48x25GbE switches on the edge,
| which are pretty economical now.
|
| I don't see this card being that popular in that market
| however; if you want solid tcp offload and asic acceleration
| there's xilinx cards with a good reputation already.
| zamadatix wrote:
| There are more markets than homelab and hyperscale data
| center, this is solid for software network services at the
| edge where cost is a concern and flexibility is a plus.
| MikroTik tends to fill these kind of niches at a cost
| competitive price point, they don't aim to sell just to
| consumers or realistically compete with established vendors
| in the high end segments, just those niche cases they think
| they can be a low cost option where there wasn't one before.
|
| My hope (once I can actually get my hands on one) is this can
| integrate well for us by offloading a lot of the routing and
| NAT type functions for a managed service network offering
| software based box we sell that handles all of the "smart"
| network functions at the site + acts as the egress point.
| Melatonic wrote:
| I think for niche portable use cases this could be very cool
| or anywhere you are super space constrained.
|
| I agree with you on most points though - and finding good
| people who know how to even use RouterOS seems like it would
| be a pain for companies as well.
| iso1210 wrote:
| I've got a few hundred mikrotiks, mainly CCRs and 1100AHs, I
| guess I could merge my monitoring machine and my router, and
| it's handy if I just want to deploy a single device somewhere
| but manage it in the same way (firewalls, vpns etc), it's
| certainly not something I've being waiting for.
| core-utility wrote:
| It's also worth saying that Mikrotik is a common platform for
| "homelabbers" who use enterprise-grade (ish) hardware in their
| homes. RouterOS isn't without its flaws and pain points, but
| Mikrotik brings high quality features into a low cost package
| that appeals to many. It's the lesser-known (and polished)
| brother of what Ubiquiti used to be.
| mjochim wrote:
| Are you saying lesser known and less polished or lesser known
| and more polished?
| bombcar wrote:
| It's lesser known and not as slick (polished) but it is
| quite capable and a good deal at the price point.
|
| And it doesn't have cloud dependencies to manage it.
| stragies wrote:
| Does anybody know, if Openwrt for this is
| planned/feasible/complicated/...? ARM64 sounds like basic boot
| could be easy, but the CPU name (AL52400) top search hits are
| from the Mikrotik product page. Is something known about the rest
| of the components?
___________________________________________________________________
(page generated 2022-03-28 23:00 UTC)