[HN Gopher] Introduction to D3
___________________________________________________________________
Introduction to D3
Author : kickout
Score : 272 points
Date : 2020-03-03 17:55 UTC (5 hours ago)
(HTM) web link (observablehq.com)
(TXT) w3m dump (observablehq.com)
| kickout wrote:
| Any other libraries (that are as low of level) as d3.js? Is D3
| still being used heavily in production by people's experience?
| stared wrote:
| I used to use D3.js as the default go-to for data. Now I start
| with Vue.js, and use D3.js only inside some components when I
| need, animations between transitions.
|
| D3.js, as someone else already noted, is "jQuery for data". And
| yes, it has its good parts, but the same as for jQuery) the
| problem is with:
|
| - mixing static HTML/SVG with generated
|
| - no modularization (every modularization is custom)
|
| For the latter, while it may not be a big deal for small
| projects, for bigger it sucks.
|
| For big stories, I created a game in D3.js. Modularization
| was... well, argument why I rewrote it a few years later to
| Vue.js.
| suyash wrote:
| Vega lite comes to mind
| kulahan wrote:
| I started a new project using it for a project I was working at
| my last job ~ a year or two ago. I got a few hits from
| recruiters interested in it once I added it to my resume.
|
| I liked it quite a bit. It's pretty fast and decently easy to
| use. Plus, all the examples Mike Bostock has thrown around
| online really helped for giving me examples of how to do
| various visualizations I was interested in.
| simonsarris wrote:
| How low level? I make GoJS, a powerful diagramming library (the
| focus is more on diagrams and interactivity, than data). It's
| low level in that you can make a lot of different things with
| it, but higher level in that it has things like an undo
| manager.
|
| https://gojs.net/latest/index.html
| paulgb wrote:
| D3 is still a useful library for visualization even if you
| replace the DOM manipulation with React, Vue, Svelte, etc. I
| think the trend is to use D3 as a library but use a more modern
| framework to manage the DOM.
|
| I have a browser extension (treeverse.app) where the entire UI
| is D3. I probably wouldn't write it that way today, but the
| codebase dates back to ~2014 and it's held up fine.
| thibautg wrote:
| I also mostly use d3 for its powerful helper functions
| nowadays: scales and axes, colors, svg path generation,
| geographic projections, array functions like d3.group
| (formerly d3.nest), csv/tsv parsing etc.
|
| I tend to leave DOM manipulation to Vue.js unless I need to
| animate transitions between states. I find it easier to
| reason about binding data to an html/svg template with Vue
| than to mess with d3 data joins and enter/update/exit
| pattern.
|
| d3 is and stays an impressive piece of work.
| sailfast wrote:
| Used in production? Yes.
|
| Alternatives at a low level? Hard to say - it really does allow
| you to do a lot at a very low level if you want BUT...
|
| - Vega Lite / Vega
|
| - HighCharts (paid)
|
| - ChartJS
|
| - Raphael (unsure if this is still used as much)
|
| - Leaflet / Turf for GIS visualizations
|
| - Server-side? PyGal? http://www.pygal.org/en/stable/
|
| If you don't want to go so low-level there are a huge number of
| D3 abstractions that allow you to pick your chart and work with
| the data. Britecharts from EventBrite is one example of an
| actively maintained abstraction.
| http://eventbrite.github.io/britecharts/
| acomjean wrote:
| Vega is low level, but built on top of D3.
|
| https://vega.github.io/vega/about/vega-and-d3/
|
| Vega lite is what I use as my day to day web graphing driver
| (it exports to svg which is awesome...)
| lmeyerov wrote:
| We end up using some of the d3 helpers, like on axis &
| scales, but go straight to chart libraries + custom
| DOM/SVG/webgl for the rest, and normal React framework for
| interactivity. The d3 functional helpers are still real gems
| and IMO under-appreciated. I think it was rewritten a few
| years to help expose those better (e.g., for tree shaking.)
|
| As soon as you go to react & friends, most of the d3 model
| does become redundant, and Yet Another Thing to burden the
| team. There's still a temptation to reuse beautiful d3
| gallery items posted by others, but generally, they're
| annoying to patch up & now you must maintain them. Better to
| use dedicated _maintained_ libs if that 's what you want.
| There have been attempts to do Standard D3 Chart libs, but
| afaict it's a graveyard of abandonware, and I approach most
| efforts here with strong caution. (The trend for the last few
| years has been each bigco to release their own NIH framework
| and then abandon as the authors leave, and little incentive
| for D3 design consultants to maintain old projects.)
| sailfast wrote:
| What "straight chart libraries" do you use? I'd be
| interested in digging into that further.
|
| I agree that React / Redux state can take the place of a
| lot of the D3 models. That said, it's nice that D3 handles
| entry / updates and changes so you can point to redux and
| it "just works" and all the transitions, etc are handled as
| your app state is handled.
| lmeyerov wrote:
| At this point... I think we only have Susie Lu
| (Netflix)'s annotator, but good chance of it being
| removed. For example, we had to do our own timeline to
| get decently fast zoom rebinning on modern screen widths
| that we abandoned a JS widget here, and histograms ended
| up being so custom that we'd get little from a library.
|
| The shortlist for GIS is DecKGL. Graphistry gets a lot of
| requests here, so we helped the DeckGL folks on some
| Arrow/GPU stuff and we had promising experiments as part
| of it, but (if I remember right) we hit memory issues and
| something else. We'll be evaluating that for GIS next
| time it makes sense.
|
| +1 to others on HighCharts team!
| mrlala wrote:
| ECharts has been very nice, it's free and open source
| https://echarts.apache.org/en/index.html
| duderific wrote:
| Bonus is that it uses almost the identical APIs as
| HighCharts. We switched from HighCharts to eCharts at my
| work and we've been happy with it.
| kickout wrote:
| Thanks for this. Never seen HighCharts. Looks nice (pending
| $$$)
| vosper wrote:
| We use Highcharts at work, and have for years. It's a
| really good library, definitely worth the money. We had a
| recent adventure with building new charts in Victory, but
| performance was not good and we came back to Highcharts and
| removed all our Victory code.
|
| We're still happy with that decision :)
| inferiorhuman wrote:
| HighCharts is very nice if you don't want to invest the
| time into honing your d3 skills.
| suyash wrote:
| Link to the actual course that mentions all the details :
| http://vis.csail.mit.edu/classes/6.894/
| boringg wrote:
| I really enjoyed d3 for the most impressive data viz products.
| However I always found the overhead to get d3 going and piping
| data to the explicit locations to be burdensome for most
| applications. My sense was that you would use d3 as a final
| polishing step for any data viz project/product or if you wanted
| to make a data product that had complicated requirements/display
| needs. I would be curious to hear if anyone uses it for data
| exploration or if the overhead has been lowered over the last
| couple years (its been 3 years since I've used it).
| Vaslo wrote:
| Used D3 for a grad data viz course project a year or two ago.
| Extremely powerful but unless you are doing custom visualizations
| or are excellent in JavaScript and Visualization it's a bit
| overkill. Much easier to do monthly reporting or even one off
| stuff in Tableau, Power BI, or the like. Tableau or Power BI I
| could pass to other analysts and without experience they could
| figure it out. If I sent them my d3 code they would cry.
|
| Again if doing advanced visualizations for a large newspaper,
| commercial presentation, it's extremely customizable and makes
| some really beautiful charts.
| boringg wrote:
| Completely agree with this statement. Wish it was implementable
| similar to a tableau or something for more universal appeal --
| or alternatively it had enough easily default applications with
| the ability to go much more custom if you needed/wanted to.
| sailfast wrote:
| Using a non-proprietary platform also has the distinct
| advantage of supporting whatever APIs you want to use and not
| having to deal with extracts of data. Having to wait for one of
| these platforms to support JSON from a web URL is really quite
| silly when you could write the query quickly (in python or
| whatever) and pass the values to your view.
|
| Or, you know, use PyGal or another server-side charting
| library: http://www.pygal.org/en/stable/
| inferiorhuman wrote:
| _Or, you know, use PyGal or another server-side charting
| library:http://www.pygal.org/en/stable/ _
|
| Is there much of a market for server-side rendering? I've
| been putzing around with a d3-inspired charting library in
| Rust mostly as a brain teaser though.
| Mountain_Skies wrote:
| Aren't Tableau and PowerBI both expensive for anything other
| than the most trivial visualization scenarios?
| warpech wrote:
| Technically, D3 is a JavaScript library, but in reality it is
| much more than that. In this article, it is called "visualization
| grammar". I have heard it being called "jQuery of diagramming",
| "declarative DSL for data visualization" and so on.
|
| Since 2011, there were already numerous cycles of the best of
| breed JavaScript SPA frameworks and libraries (I'm talking
| everything from jQuery, through AngularJs, to React), but in data
| visualization, D3 seems to hold its position very well.
|
| What are the things that D3 did right to get the acclaim, and why
| does it keep it for so long?
|
| D3 is approaching 10 years since the initial release, did it
| stand the test of time and will it keep its power for the next 10
| years?
| beetwenty wrote:
| D3 does the Right Thing with respect to dataviz.
|
| That means: it's conceptually overloaded for doing quick-and-
| dirty conventional charts, where you just want to plug in a few
| parameters and have it "just work" - but excellent once you
| need to customize and make it work with your specific
| requirements.
|
| That it has the concepts, and a clear notion of them, is the
| critical difference. Most libraries, most of the time, don't
| add new concepts, they just have a premade black box of
| features and functions. Sometimes you want a premade black box,
| but often you want to open up the box shortly afterwards, and
| that creates the inevitable trend towards either remaking it as
| your own box, or being one of hundreds of people who gradually
| grow it into a monstrosity that does everything.
|
| But a library that is concept-focused doesn't have to get that
| much bigger: it's just another kind of interface, like a
| programming language or an operating system, and that puts it
| on a more sustainable track.
| IshKebab wrote:
| Vega calls itself a "visualisation grammar" and I think it is
| much more worthy of that title. I just wish it was efficient.
| Sadly it represents every data point as a JavaScript object. In
| other words it uses array-of-structs style rather than struct-
| of-arrays which is an obviously bad choice in JavaScript if you
| want efficiency.
| warpech wrote:
| Interesting what you say about struct-of-arrays, I have been
| following this topic closely recently. Do you know any well
| known JS code implemented using struct-of-arrays or have some
| interesting resources about it? I am worried that SoA
| codebase is difficult to maintain and hard to express in a
| simple API.
|
| I have been reading two resources about it,
| https://ourmachinery.com/post/data-structures-part-1-bulk-
| da... and https://stackoverflow.com/questions/39799874/why-
| does-javasc....
| foota wrote:
| It's in a different domain, but you might be interested in
| unity (the game engines) data oriented stack. Basically it
| serializes properties attached to an entity, which you then
| write systems that map against entities with certain
| combinations of properties attached. Seems like this could
| be amenable to JavaScript as well.
| mbostock wrote:
| Here is a notebook on the subject. I'd also check out
| Apache Arrow.
|
| https://observablehq.com/@mbostock/manipulating-flat-arrays
| TeMPOraL wrote:
| Thank you for that. For the first time I finally
| understood what the Apache Arrow is - until today, I
| didn't realize it's just a way to do SoA in different
| languages + a lot of buzzwords.
| bauerd wrote:
| Declarative data binding?
| jcranmer wrote:
| > What are the things that D3 did right to get the acclaim, and
| why does it keep it for so long?
|
| Once upon a time, there was an infovis library for Java called
| Prefuse. Then the developers abandoned that library for a Flash
| infovis library called Flare. A JS library called Protovis was
| inspired by Flare, and then Protovis itself was replaced with
| D3.
|
| I've used 3 of those libraries (never wrote any Flash code).
| The only one I would actually recommend people use is D3--it's
| literally the first (honestly, only) infovis toolkit I've used
| that doesn't feel like pulling teeth to get anything done.
|
| What D3 does differently from its lineage is that it is data-
| centric. Each datapoint is an object, corresponding to a DOM
| node, and the layouts in D3 don't actually draw anything; they
| just set properties on the object--you're responsible for
| actually converting those into setting SVG properties for
| display.
|
| It sounds like this would be a nasty headache, but this way is
| actually much better than the traditional way of handling
| things (where the visualization is essentially treated as a
| display widget in a normal GUI toolkit). Any time you want to
| progress beyond a basic "display a graph with static data," you
| start to need a lot more fine-grained control over display
| elements. Want to link multiple displays of the same dataset,
| so clicking on one in one screen highlights them in all the
| others? That's _very_ hard in prefuse, but quite easy in D3.
| moron4hire wrote:
| In my mind, D3 is a DSL for ingesting structured data and
| outgesting DOM. While it has certainly been a few years since I
| touched it last, what I remember was that it basically didn't
| touch anything in the data or DOM, it just made the piping
| between the two super easy. So I think it was that approach:
| not introducing new idioms for existing things, but instead
| introducing the pipeline approach, with a very simple syntax,
| all made it very amenable to learning very quickly and rapidly
| iterating an idea, tweaking it until you get just the thing you
| want.
| inferiorhuman wrote:
| _What are the things that D3 did right to get the acclaim, and
| why does it keep it for so long?_
|
| d3 (especially now that they've broken the bits and bobs into
| discrete packages) takes a very unixy philosophy: do one thing
| and do it well. d3 lets you iterate over your data and get
| useful items back (e.g. scales that you can use to create
| geometric shapes for a chart). It's intimidating for some but
| offers a ton of flexibility you don't see with other charting
| libraries.
|
| Mike Bostock also put a lot of effort into making sure that d3
| lets you create beautiful charts.
|
| Every other charting library I've seen takes a very different
| approach. Often enabling you to create ugly, inflexible charts.
| appleiigs wrote:
| In addition to breaking it up into discrete packages, the
| devs are also OK with improving things even if it includes
| breaking changes.
| burticlies wrote:
| They kept it really low level and didn't make many opinions on
| how to join the dots. There are functions to manipulate data,
| functions to make scales, and functions to and functions to
| render svg (plus dom bindings). These decisions make individual
| charts quite verbose, but the trade off is you can visualise
| anything you want. Just look on npm how many charting libraries
| use d3 as a base.
| tenaciousDaniel wrote:
| This is true. We decided to use D3 directly to render charts
| at my job, and it was honestly kind of a terrible decision.
| Ideally you would use a library that uses D3 to render
| charts.
| datashow wrote:
| I had the same experience. It started great when you just
| want a simple line chart, but when you need more and more
| features, it will be like creating tons of abstractions and
| writing a whole charting library ...
| inferiorhuman wrote:
| What made d3 a terrible decision?
| rolltiide wrote:
| each D3 project is an entire application of its own
|
| adding the "simple pie chart" to your view is not simple
| at all.
|
| It needs independent UX planning, system architecture
| planning, and multiple sprints to implement
|
| All the example code and prebuilt charts are outdated due
| to using old versions of D3 or other dependencies and
| essentially have to be rebuilt from scratch
| duderific wrote:
| My guess is that d3 is too low-level to make reuse
| simple, and that using a higher-level abstraction would
| make the code more maintainable.
| inferiorhuman wrote:
| _My guess is that d3 is too low-level to make reuse
| simple, and that using a higher-level abstraction would
| make the code more maintainable._
|
| In what context? It's not that hard to collect some
| scales and axes into a JS object, mix it with judicious
| use of CSS et voila.
|
| I understand that d3 is low-level and verbose, but I
| can't say that I've any regret over deploying it in
| production.
| tenaciousDaniel wrote:
| It's too low level. For reference, I just re-implemented
| the entire thing in React. _just react_ , with svg
| elements directly. Not using D3 at all. And it was
| roughly the same level of effort.
|
| edit: saying that D3 was a terrible choice is _not_
| saying that D3 itself is terrible. D3 is amazing. But
| before choosing it, you should know exactly what it is,
| and whether it helps in your specific use case.
| xyz0 wrote:
| That's the idea when you're using D3 with React: React
| handles the DOM updates, D3 handles data wrangling and
| math. You said you reimplemented your charts in SVG
| directly - did you also reimplement scales, chart axis
| generation, d3-collections, d3-time, etc?
| nightski wrote:
| I'm not sure how you'd implement scales, color theory,
| etc.. in React (a rendering library). There are a lot of
| useful parts of d3 which have nothing to do with
| rendering to the DOM. In fact, React and D3 complement
| each other quite well (maybe that is what you were
| saying?).
|
| D3 is slightly more verbose than the one liners, but with
| that I get complete control over the scales and
| appearance of my chart. In addition it's trivial to
| combine multiple charts in a single diagram.
| curiousgal wrote:
| I think of it as the assembly of data viz.
| [deleted]
| jonathankoren wrote:
| Maybe I'm the minority, but whenever I worked with D3 to do
| anything beyond the simple barcharts this shows, I ended up
| having to manipulate the SVG DOM manually, which quite frankly
| sucks.
| mkchoi212 wrote:
| Cool library! I like it when libraries don't have too many
| configurations set to a default.
|
| e.g. https://github.com/danielgindi/Charts has so many default
| options set for a graph that 80% of code is doing something like
| chart.option.isEnabled = false
| factsaresacred wrote:
| I switched from D3 to echarts[0] and never looked back. Still
| powerful and customizable but comes with a much easier API to
| reason about.
|
| [0] https://github.com/apache/incubator-echarts
| stared wrote:
| As much I try to like ObservableHQ, this "updates happen above
| the code, sometimes a bit far, so you if you scroll too much, you
| don't see them" is one thing that makes it less intuitive than
| Jupyter Notebook style.
|
| It took me some time to realize that running cells actually
| change something in the country list.
| pphysch wrote:
| It's not just a Jupyter Notebook But In JS, though. Dynamically
| interacting with the DOM is perhaps the core feature of
| Observable, while Jupyter/iPython very much embraces the strict
| imperative style (ipywidgets etc being more a nice hack than
| anything)
|
| While I do think there could be UX improvements, I don't think
| this is a flaw in Observable.
| stared wrote:
| I think that in JS there is much more potential for
| interaction (as frontend IS JavaScript; for Python and other
| languages, it is this language + JavaSript). Yes, ipywidgets
| are hacks (as many other interactive things).
|
| So, I am surprised that there is no standalone REPL in that
| style. The closest things I've found are ObservableHQ and
| RunKit (the other is much more Jupyter-like, but still -
| AFAIK runs code in the backend).
| kickout wrote:
| Always viewed this as a good feature, not a bug
| TeMPOraL wrote:
| It's a feature, not a bug. ObservableHQ notebooks are
| _reactive_ - there 's a DAG underneath. It's not a Jupyter-
| style execution log.
| K0SM0S wrote:
| > _there 's a DAG underneath_
|
| Would you or someone care to elaborate? Do we mean the
| 'linking' between 'nodes' (cells) as an interpreter/compiler
| would do over a file/object structure?
|
| Thus I assume, opening a world of options e.g. for
| vectorizing performance, type checking and all?
|
| If so, we have in one such notebook a true slice of "visual"
| IDE the kind of which Microsoft could only ever dream about!
| (so far) ;-)
|
| [Side-related note: I'm amazed at the emergence of the
| notebook paradigm over the last 10 years, accelerating for at
| least 2-4 now. See how they do it at Netflix. There's a case
| to be made that the notebook paradigm could really bridge
| wide open the "programming rift" between nerds and, well,
| everybody else, at least in skill-driven professional
| contexts. There's a short way from here to a slew of clever
| domain-driven script languages plugging straight into BI.]
___________________________________________________________________
(page generated 2020-03-03 23:00 UTC)