[HN Gopher] Build Your Own Redis with C/C++
___________________________________________________________________
Build Your Own Redis with C/C++
Author : ibobev
Score : 68 points
Date : 2023-03-18 20:11 UTC (2 hours ago)
(HTM) web link (build-your-own.org)
(TXT) w3m dump (build-your-own.org)
| TheChaplain wrote:
| For giggles I was watching this thread waiting to see how long
| until the word "rust" appeared. Didn't take long :^)
|
| Someone on reddit joked that Rust is becoming the new Crossfit
| meme;
|
| "How can you tell when someone programs in Rust?" "Don't worry,
| they'll tell you."
| noncoml wrote:
| You know what? If I was more confident in my coding abilities,
| or some seriously good static analysis tool was available for C
| or C++, I would program in one of those languages.
|
| I'm really not big fan of Rust, but I love that's it's always
| looking over my shoulder for stupid mistakes.
| throwaway5959 wrote:
| It's OK. The truth is that even seasoned C/C++ developers
| can't write safe C/C++ code. Otherwise exploits wouldn't
| exist and static analysis wouldn't be necessary.
|
| What are you building with Rust?
| noncoml wrote:
| I've been programming professionally in C since 2005 and
| still short myself in the foot all the time.
|
| I'm Rust I'm found various side projects. An all in one
| PiHole like dns server, a distributed key value store based
| on raft and an openwrt module.
|
| But nothing interesting or revolutionary enough to share.
|
| How about you?
| 1024core wrote:
| > The truth is that even seasoned C/C++ developers can't
| write safe C/C++ code.
|
| Except DJB, of course.... ;-)
| josephg wrote:
| I love C and Rust - I've been using rust full time for the
| past couple of years.
|
| But if I wanted to let loose these days and write some code
| without worrying about the safety that rust has to offer, I'd
| reach for Zig. It seems to solve a lot of C's problems while
| being a much cleaner, well thought through language for low
| level code. Im a fan!
| mattrighetti wrote:
| I would really love to take a look at this book but I'm not
| really interested in C/C++, has anyone tried to follow along but
| with another language? How did you find it?
| [deleted]
| jvans wrote:
| I have done something similar with ruby, this is a great book:
| https://workingwithruby.com/downloads/Working%20With%20TCP%2...
|
| I would recommend doing C at some point in your career, nothing
| comes close in terms of forcing you to understand the hardware
| you are using.
| sitzkrieg wrote:
| id recommend assembly language to really achieve this. the
| further away you get from x86 the more enjoyable it seems to
| be fore a human. skilldricks easy6502 is a really good start
| Mike_12345 wrote:
| That was true in the 70's but C doesn't require that you
| understand much about modern hardware.
| https://queue.acm.org/detail.cfm?id=3212479
| doodlesdev wrote:
| It still is more low level than any other language
| currently (that can be run on multiple architectures) which
| I think is what the OP was saying. If you compile C without
| compiler optimizations, it will generate code that is
| exactly what you wrote. The argument the paper makes is
| that the instructions available in modern processors and
| the way we use them to optimize code (such as instruction
| level parallelism) is a consequence of C being so important
| despite being made for much older and much simpler
| machines. Because if the generated binaries from any C
| compiler without optimizations isn't low level then you
| might as well say the same about x86 assembly, and then
| you're basically out of options.
| Mike_12345 wrote:
| But it's not. It has become a high level language. The
| idea that C is still a low level language is no longer
| true. It is highly abstracted from modern hardware.
|
| Edit: I just saw your edited comment. Yeah the point is
| you really don't need to know much about the hardware to
| write C. It doesn't force you to understand what's
| actually going on behind the scenes.
| doodlesdev wrote:
| Yeah sorry about the ninja edit. Very often I reply to a
| comment without actually finishing and then edit it. I
| thought Hacker News was delaying my comments by 1 minute
| though according to my configs? Anyways, I agree that you
| don't really need a deep understanding of the hardware
| nowadays to write C, but my point is that it's still
| useful to learn because you still need more understanding
| of the hardware than you need to write something such as
| Ruby.
|
| The point the OP was making is that it still forces you
| to learn more about the hardware than other languages.
| Arguably, it also forces you to learn more about the
| software itself with things being much more explicit than
| in a modern complex language such as Rust. I'm also just
| not aware of _any_ language that actually maps well to
| what the CPU is doing nowadays since they are such
| complex pieces of silicon.
| Mike_12345 wrote:
| But does it really force you to learn more about the
| hardware versus other languages? I am challenging that
| assumption or belief.
|
| C has memory pointers, but those are pointers in a flat
| memory space abstracted over the physical memory
| hierarchy.
|
| So yes, you do need to understand that the hardware has
| memory addresses and that pointers can reference memory
| addresses. Aside from that I don't see much difference to
| Java or Python in terms of requiring a deeper
| understanding. Even Python has bitwise operators.
| Gibbon1 wrote:
| My take is both C and the hardware are targeting the same
| abstract machine. It seems too that attempts shift things
| one direction or another haven't been successful.
| Itainium which tries to force the compiler to deal with
| instruction scheduling and parallelism, failed. Things
| like LISP machines also didn't do well.
|
| So thinking in terms of the abstract machine are valid
| for now. The exceptions mostly have to do with caching
| and understanding that processor can and does execute
| short sequences of instructions in parallel.
| _gabe_ wrote:
| This whole article seems to be picking a lot of nits. I
| didn't read it too deeply, so feel free to correct me if
| I'm wrong, but the biggest complaints highlighted in the
| article are:
|
| 1. Modern CPUs use instruction level parallelism (ILP)
|
| 2. Memory isn't linear (you have separate caches, L1, L2,
| L3 and main memory)
|
| If you've ever debugged a C or C++ program in release,
| you've quickly found out about ILP. The code still maps
| relatively close to the hardware, it just won't run in the
| sequential order you've provided it in. Many C and C++
| programmers know this and try to make the implicit
| assumptions in their code explicit to allow the compiler to
| reorder instructions more easily/reduce memory dependency
| chains[0].
|
| And I'm sure you've heard _several_ proponents over the
| past few years (Mike Acton comes to mind) espousing data
| oriented design. This is an entire code methodology
| intended to help the CPU caches, and it shows an implicit
| understanding that memory is not linear. Heck, most C /C++
| programmers realize that they're using VRAM all the time,
| and the memory locations they get aren't necessarily backed
| by physical memory until the OS sorts it out. This is
| especially transparent if you've ever done any sort of
| memmapping with files or played with virtual memory.
|
| Anyways, C doesn't necessarily map directly to the
| hardware, but it's a heck of a lot easier to intuit what a
| C program will end up doing on the CPU vs what a Python
| program will do. And most C/C++ programmers realize this
| fact, and actively write code to utilize the fact that C
| does not map directly to hardware.
|
| [0]: https://johnnysswlab.com/instruction-level-
| parallelism-in-pr...
| josephg wrote:
| Yeah. Also, learning C is still probably the best way to
| learn how raw pointers work. And pointers underpin
| everything - even if you spend your life in Python or
| Java.
|
| When I was teaching programming, it was always a rite of
| passage for my students to implement a linked list in C.
| Once it clicked for them, the world of programming opened
| up.
|
| C is also still the universal glue language for FFI.
| Wanna call Swift from Rust? You can always reach for C.
| iudqnolq wrote:
| if you're interested in Rust this might help. It's intended to
| be an introduction to the tokio async runtime for Rust by
| having you implement a minimal redis.
|
| https://tokio.rs/tokio/tutorial/setup
| [deleted]
| tylerhannan wrote:
| I sort of want, one day, to read the same article targeted at
| Delphi or Turbo Pascal. lol.
| FpUser wrote:
| All styles of programming for implementing Redis are supported
| in Delphi. It'll work just fine. As for Turbo Pascal - replace
| it with FreePascal/Lazarus which together comprise opensource
| version of Delphi.
| andsoitis wrote:
| > lol
|
| What's the joke? I don't get it.
| sbmthakur wrote:
| Codecrafters has a similar exercise:
|
| https://app.codecrafters.io/courses/redis?track=c
| intelVISA wrote:
| No C++? Hard pass from me.
| JTyQZSnP3cQGa8B wrote:
| It seems to be a nice project but they should remove "C++" from
| the name since both languages have nothing in common anymore,
| especially the use of smart pointers and Boost.Asio compared to
| "malloc and socket" that is used in the book.
| hewlett wrote:
| You can still use raw pointers and sockets in C++, which the
| code examples provided in the book do
| JTyQZSnP3cQGa8B wrote:
| I agree but no one calls this C++ anymore especially since
| C++14/17, and I fail to see why you would use a C++ compiler
| for such a code.
| [deleted]
| gpderetta wrote:
| Well, if you are implementing something like ASIO, and of
| course c++ is a good language for that, you'll be dealing
| with sockets directly.
| Arelius wrote:
| I'm starting to understand there are clearly 2 different
| worlds, if you live in one where C++ assumes the use of smart
| pointers, or especially Boost.
|
| In the C++ world I live/work Boost is avoided like the plague.
| I always wondered how Boost continues to exist, but it seems in
| certain communities it's held up as a standard to strive for.
| heywhatupboys wrote:
| > C/C++
|
| there is _NO SUCH THING_
| user5534762135 wrote:
| "Build your own Redis with Rust/Fortran"
|
| If the author actually can't understand the difference between C
| and C++, there's a pretty low chance of any good code being in
| the book. If they do, they should keep an eye on the editor next
| time they go into empty marketing mode for the title.
| noncoml wrote:
| I kid you not I once had a recruiter asking someone with
| experience in C/C+/C++
|
| Not a typo for C#. He really wanted someone with experience in
| C+
| heywhatupboys wrote:
| benefit of the doubt: C with classes?
| noncoml wrote:
| Maybe. Who knows what his client asked. To me he confirmed
| they were looking for C+...
| josephg wrote:
| I've lost count of the number of recruiters who shorten
| Javascript to Java. Those are not the same thing! Arghhhhh
| matttb wrote:
| > The code is written as direct and straightforwardly as the
| author could. It's mostly plain C with minimal C++ features.
|
| Sounds fair to me
| sitzkrieg wrote:
| this is also one of my biggest pet peeves too
| mzimbres wrote:
| I haven't created Redis itself but a Redis client library. It all
| started with a chat server I wanted to write on top of
| Boost.Beast (and by consequence Boost.Asio) and at the time none
| of the clients available would fill my needs and wishes
|
| - Be asynchronous and based on Boost.Asio.
|
| - Perform automatic command pipelining [1]. All C++ libraries I
| looked at the time would open new connections for that, which
| results in unacceptable performance losses.
|
| - Parse Redis responses directly in their final data structures
| avoiding extra copies. This is useful for example to store json
| strings in Redis and read them back efficiently.
|
| With time I built more performance features
|
| - Full duplex communication.
|
| - Support for RESP3 and server pushes on the same connection that
| is being used for request/response.
|
| - Event demultiplexing: It can server thousands of requests (e.g.
| websocket sessions) on a single connection to Redis with back
| pressure. This is important to keep the number of connections to
| Redis low and avoid latency introduced by countermeasures like
| [3].
|
| This client was proposed and accepted in Boost (but not yet
| integrated), interested readers can find links to the review here
| [2].
|
| [1] https://redis.io/docs/manual/pipelining/
|
| [2] https://github.com/boostorg/redis/issues/53
|
| [3] https://github.com/twitter/twemproxy
___________________________________________________________________
(page generated 2023-03-18 23:00 UTC)