[HN Gopher] Phlare: open-source database for continuous profilin...
___________________________________________________________________
Phlare: open-source database for continuous profiling at scale
Author : PaulWaldman
Score : 177 points
Date : 2022-11-02 15:28 UTC (7 hours ago)
(HTM) web link (grafana.com)
(TXT) w3m dump (grafana.com)
| kapilvt wrote:
| its a db, anything for ingesting.. ideally an open telemetry
| ingester?
|
| [update] hmm.. per docs wants pprof format and an agent.
| https://grafana.com/docs/phlare/latest/operators-guide/confi...
|
| language support
| https://grafana.com/docs/phlare/latest/operators-guide/confi...
|
| re multi-lang support via pprof - afaics python hasn't been
| touched in years, java is shiny new albeit also first party, in
| golang pprof is native, and rust pprof seems active.
| RedShift1 wrote:
| What about maintenance on the existing projects? Open issues on
| Github:
|
| Grafana: 2.6k issues, 275 PRs
|
| Loki: 531 issues, 113 PRs
|
| Mimir: 305 issues, 40 PRs
|
| Tempo: 159 issues, 19 PRs
| [deleted]
| mrtweetyhack wrote:
| [deleted]
| Rperry2174 wrote:
| I suspect this is partially because internal efforts and
| agendas take priority over open source community at this point.
|
| For example, I created an issue requesting a flamegraph
| visualization in grafana[1] and now it makes sense that they
| didn't initially respond because they were building it
| internally in secret and didn't want to spoil the big reveal
| (when they did respond they did mention that it was a secret).
|
| They're also less incentivized now to tend to issues and PRs
| that help others outside of their ecosystem (i.e. competing
| logs, metrics, tracing, profiling, etc products).
|
| [1] https://github.com/grafana/grafana/issues/53723
| PaulWaldman wrote:
| >Sorry for this being a bit secret until now but we tried to
| quickly build and iterate something usable. There isn't much
| documentation for it as it is behind feature flag it's not a
| GA feature but hope that will come soon.
|
| Secrets kinda conflict with the whole Open part of OSS.
| onedr0p wrote:
| While they are open-source projects, I bet their software is
| still driven by customer feedback. I wouldn't put it past them
| to prioritize paying customer requests which takes time away
| from implementing features, doing bugfixes or code review.
| madeofpalk wrote:
| I don't think it's fair to say that paying customer feedback
| is prioritised at the expense of the open-source community.
|
| The thing with the core Grafana product being open source is
| that there's not that much dissimilarity between
| paying/enterprise Grafana users, and open-source users.
| Feedback from one set will almost always work in favor for
| the other.
| shamiln wrote:
| Is there any observability product that Grafana Labs doesn't
| produce?
| [deleted]
| posnet wrote:
| Would love to see this integrate with magic trace [1]. I'll need
| to look at the code for the flamegraph plugin, because handling
| nanosecond timestamps in flamegraphs seems to break most tools
| due to float precision.
|
| (1) https://github.com/janestreet/magic-trace
| mdaniel wrote:
| > Make sure the system you want to trace is supported. The
| constraints that most commonly trip people up are: VMs are
| mostly not supported, Intel only (Skylake2 or later), Linux
| only.
|
| I'd guess this rules out AWS as well as containers, too, right?
| posnet wrote:
| It works on metal instances.
| dang wrote:
| Another ongoing thread, presumably related:
|
| _Grafana Faro: An open source project for front end application
| observability_ - https://news.ycombinator.com/item?id=33439799 -
| Nov 2022 (2 comments)
| candiddevmike wrote:
| Why can't one storage thing be used for everything instead of
| disparate datastores? I need Loki for logs, Tempo for traces,
| Prometheus for metrics, and now Phlare for profiling. Three of
| those are using object storage under the covers, why not one
| datastore to rule them all?
| beebmam wrote:
| There are (vastly) different stream compression techniques
| based on object type, optimized per scenario.
| stormbeard wrote:
| This can't be the reason- lots of datastores let you
| customize the compression scheme of various subsets of data.
| beberlei wrote:
| MySQL had the fix for this many years ago, storage engines :)
| chrnola wrote:
| See also: https://pyroscope.io/
| mnutt wrote:
| Other than a reference to ebpf at the end, it wasn't super clear
| to me where the profiles come from and what they support. Phlare
| itself is just a database for storing/querying profiles, right?
| PaulWaldman wrote:
| >Phlare itself is just a database for storing/querying
| profiles, right?
|
| Sounds like it, with the addition of a Grafana panel. It seems
| like there is a bit of overlap between this and the other
| products like Tempo, Loki, and Mimir. This graphic seems to
| indicate it stands independently though, aside from Grafana
| visualizations.
| https://grafana.com/static/assets/img/diagrams/grafana-diagr...
| mnutt wrote:
| Elsewhere they reference making updates to the Grafana Agent
| in the future, but it's hard to say what "scrape profiles"
| means.
|
| But I really, really like the idea. Often when I want to test
| the performance of a change I'll launch test and control
| canary instances with a small percentage of live traffic, run
| perf against each, collect the data, load it into a local
| https://profiler.firefox.com/, and try to compare the
| differences. It would be awesome to automate that process.
| Beyond that, I often keep notes about the tests but the
| profiles themselves are a real pain to store and catalogue.
| gouthamve wrote:
| Currently Phlare supports pprof which exposes profiling
| data on a http endpoint.
|
| We need to pull these profiles in at a regular interval by
| hitting the HTTP endpoint and we call this "scraping
| profiles".
|
| This is very similar to how Prometheus.io works.
| dig1 wrote:
| Thanks Grafana team, but no thanks. I've been using Grafana for
| years, but I started to be cautious with them, when they began to
| shift things around, probably due VC pressure for more $$$ or
| internal bureaucracy. First, they change the license to AGPL3 - I
| don't mind it, but a sudden license change (from a more open
| Apache license to a more restrictive AGPL) should raise some
| eyebrows.
|
| Second, when you go to download OSS version, they will first nag
| you with the cloud version (I'm _downloading_ that thing, not
| signing up!), then will, by default, link to the enterprise
| version. Something similar Elastic has done for years.
|
| Also, their cloud offering advertises "Free Forever" (whatever) -
| we all know how these things end ;)
| codegeek wrote:
| Honestly, I see the same pattern with all OSS products that are
| funded by VC. I guess they have to show those growth numbers. I
| don't have a problem with them trying to make money but OSS
| almost seems like a gimmick/dangling carrot to really just
| signup for their cloud version. I would rather have them be
| honest.
| [deleted]
| nobodycorporeal wrote:
| doesnt seem to work
| agilob wrote:
| I've been using pyroscope oss for about a year now. It's more
| mature, supports more agents, but isn't as interactive as Phlare.
| It integrates with Promethues and Grafana. No complaints, it's
| pretty good.
|
| https://pyroscope.io/docs/
|
| Edit:
|
| Just tried to run it in Grafana, but it's not easy. Datasource
| for Phlare is not in a stable grafana image:
| --set image.repository=aocenas/grafana \ --set
| image.tag=profiling-ds-2 \
|
| flamegraph plugin is in beta behind feature flag
|
| https://grafana.com/docs/grafana/next/panels-visualizations/...
|
| >Note: This panel is currently in beta & behind the flameGraph
| feature toggle.
|
| With these two issues in mind, announcement of this product feels
| a bit rushed just to show it during ObservabilityCON, when I
| can't run it locally with stable images and plugins. I hope to
| see it release in mainstream repos soon!
| Sytten wrote:
| Is it just me that finds the whole grafana stack so confusing.
| Too many tools not that well integrated. I dont even know where
| to start...
| PanosJee wrote:
| Have you tried netdata?
| ggregoire wrote:
| I just checked their website and they have 4 products listed
| under "Products"...
|
| Grafana is obviously the main one, their original product and
| the most popular one. It's to aggregate data from various
| sources and make dashboards.
|
| Loki is a logs collector that I personally didn't try but I
| think it's popular. They released it after Grafana.
|
| I don't know the third one.
|
| The last one seems to be a wrapper around Prometheus (a metrics
| collector/database).
|
| Fair to assume you should start with Grafana. For the source,
| if you don't have a Prometheus instance, you can test it with
| any SQL database.
| [deleted]
| ren_engineer wrote:
| I think their goal is for Grafana to be the abstraction layer
| for every datastore. You just hook up their specialized cloud
| hosted datastore for each type of data and then plugin Grafana
| and don't even think about it much.
|
| Basically they are churning out all these different projects
| that just need to be "good enough" from a performance
| perspective
| soitgoes511 wrote:
| I have used Grafana in conjunction with Influxdb (1.x & 2.x),
| postgresql, mssql and mariadb with little issue for ~5 years. I
| have not dipped my toes in the water experimenting with the log
| aggregators. Some of the 3rd party plugins have been difficult
| to use and there are some odd bugs here and there (histograms
| on older versions finding the proper xmin and xmax). What
| specifically has been confusing?
___________________________________________________________________
(page generated 2022-11-02 23:00 UTC)