[HN Gopher] US consumer spending dashboard built on data from 50...
___________________________________________________________________
US consumer spending dashboard built on data from 50M+ cards now on
Snowflake
Author : izyda
Score : 100 points
Date : 2023-11-01 18:40 UTC (4 hours ago)
(HTM) web link (app.snowflake.com)
(TXT) w3m dump (app.snowflake.com)
| dylan604 wrote:
| with all of the fintech apps that attach to a card, i wonder if
| they break down spending categories by vice. how many people
| paying their dealer or other vice related friends with cashapp,
| venmo, etc, and then bigData making the connection of what the
| transactions really were?
| izyda wrote:
| Hey there - founder here. We can indeed break down by payments
| in these FinTech apps. There are privacy concerns with certain
| aspects of this, which we of course could not touch. However,
| measuring the "shadow" economy, of course, serves a legitimate
| purpose.
| dylan604 wrote:
| you're not doing yourself any favors with the privacy minded
| folks that think people analyzing the data they did not
| provide to you directly for that purpose is something to be
| removed like cancer. but hey, at least you're honest about
| the granularity of the unintended data you can discern from
| the data you've purchased about everyday people.
| izyda wrote:
| I wasn't quite clear - we don't have access to identifying
| information in the data (ie. we could measure Venmo but not
| that John Doe paid $X to Jane Doe at phone number XXX-XXXX)
| nerdponx wrote:
| If you have individual transactions with unique
| identifiers, that's basically the same thing.
| terminous wrote:
| > analyzing the data they did not provide to you directly
| for that purpose is something to be removed like cancer
|
| For the record, cancer is an appropriate metaphor to
| describe the data broker industry
| dogman144 wrote:
| Talking to adtech about their products vs criticizing their
| products is the most illuminating thing a privacy-minded
| person could do.
|
| The nonsense "we care about privacy" wrappers go away and
| they start talking product-speak about the details. I don't
| think privacy regulations change until adtech product leads
| get comfortable enough talking publicly about their deadpan
| views on what they can find out about users.
| brundolf wrote:
| There used to be a very funny website called Vicemo that would
| scrape and display (public by default!) Venmo transactions
| containing certain keywords/emojis
|
| https://www.thecut.com/2015/02/vicemo-collects-all-your-sket...
| izyda wrote:
| Yeah interestingly, Venmo makes (or at least used to make)
| every public transaction available in their API
| xnx wrote:
| Hard to tell if this is interesting without seeing a sample of
| the data. Though the data originates from "50M+" cards, the
| summary doesn't make clear how many rows of data this is. Might
| not be that many rows since it doesn't get any finer than weekly
| granularity.
| izyda wrote:
| Hey founder here -
|
| It would be indeed great if Snowflake allowed us to preview the
| data. That said, if you have a Snowflake account, you can mount
| it and automatically get a trial (you can run arbitrary queries
| against it).
|
| The data is aggregated at a weekly level, by category,
| merchant, and demographics. It is not single individuals' data
| if that is what you are after.
| RC_ITR wrote:
| Interesting product - compared to competitors like
| SecondMeasure, what differentiates your panel/data cleaning
| approach vs. everyone else (since I assume the data is
| sourced from the same place as you competitors)?
| izyda wrote:
| I am a big admirer of what Second Measure built before
| their acquisition.
|
| - Different data sources - More accurate (though this is of
| course debateable, our plan is to publish benchmarks,
| accuracy, transparency, etc.) but this will always be
| debateable - Focus on data scientists & Snowflake users
| (rather than a SaaS platform)
|
| Obviously, this is a very early product. The key is to join
| types of datasets together, while maintaining accuracy (see
| vision outlined here:
| https://magis.substack.com/p/datanomics)
| gigatexal wrote:
| How are you getting this data? Are you paying brokers for it?
| jeron wrote:
| that's for him to know and you to find out
| supportengineer wrote:
| Who wrote the sample query? If I was trying to "Compare
| Chipotle's sales performance to that of McDonald's", my query
| would not start with "SELECT *". There would be some meaningful
| columns.
| izyda wrote:
| Yeah that's fair. The tables are all in an EAV format (narrow),
| so the number of columns coming along here is very small (you
| would filter for Chipotle/McDonalds in your WHERE) but it is
| fair that that sample query could be more instructive / better
| practice to name columns.
| ththth1 wrote:
| what's your tech stack to process this data prior to loading to
| snowflake?
| izyda wrote:
| We're doing almost everything entirely in Snowflake. Snowflake
| is our lead investor (https://www.reuters.com/technology/data-
| startup-cybersyn-rai...) and we've found it extremely helpful
| to build entirely on their technology.
|
| We're ingesting from S3 or FTPs usually.
| ZeroCool2u wrote:
| It seems odd that you can't even preview a small cached subset
| (10 rows?) of a table in Snowflake. I don't even remember a time
| I couldn't see a table preview in BigQuery, so surely it's not a
| technical limitation?
| izyda wrote:
| I definitely agree Snowflake Marketplace should have this..
| there is no technical limitation.
| bagels wrote:
| They could have baked it in to the descriptions or code
| samples.
| AirMax98 wrote:
| White screen on iOS... render error?
| izyda wrote:
| Might be a render error on the Snowflake side -- check
| https://docs.cybersyn.com/our-data-products/consumer/consume...
| instead
| elyrly wrote:
| I'm curious how cybersyn sources the data (partnership with data
| sellers? etc?) quality, reliability, and freshness are some that
| come to mind. Entry point using snowflake marketplace (series a
| investor) seems reasonable. We're missing the aha moments for
| companies when they join cybersyn datasets to their own, that
| story is hopefully in the works.
| izyda wrote:
| In general, we're sourcing from a variety of 1st party sources.
|
| But yes, the long term vision is to make this all joinable
| (https://magis.substack.com/p/datanomics)
| data_ders wrote:
| what is a "card" in this context? credit card?!
| izyda wrote:
| Credit and debit cards
| mritchie712 wrote:
| The docs are more helpful then the link above:
|
| https://docs.cybersyn.com/our-data-products/consumer/consume...
| riku_iki wrote:
| Does this dashboard cost 2k/m?..
| ricardobayes wrote:
| For those who can use this data to predict something/use it in
| a model, I guess it's pocket money. This is some very powerful
| data in the right hands.
|
| To add to that, 2k/mo is pretty much the "go-to" pricing for
| most B2B SaaS products.
| izyda wrote:
| This is correct. Note you are not getting just the dashboard,
| but you are getting the underlying data itself too -- so you
| can write arbitrary queries. That said, it is meant for large
| enterprises that have a clear path to ROI.
| void-star wrote:
| This begs the question: where is the button I can press to
| see if my personal data is included and a quick and easy
| way to inform you to remove it from your service?
| ketzo wrote:
| No personally identifying information in the dataset, so
| I imagine you would have to find some way to deanonymize
| yourself from clues in the data.
| gardenhedge wrote:
| How is it verified that the data is trustworthy then?
| hipadev23 wrote:
| $2k/mo for consumer credit card data is absurdly cheap by at
| least an order of magnitude. So it's either mispriced or it's
| garbage data.
| izyda wrote:
| It is aggregated data. You are correct that if someone were
| selling data on individual cards, it would be a lot more
| expensive.
| hipadev23 wrote:
| Aggregated or not, seeing Affinity's panel being sold for
| peanuts on snowflake was not on my 2023 bingo card.
| beefield wrote:
| Sorry a small thread hijack. I may be considering returning to my
| previous employer who has since I left started using Snowflake.
| Is that unambiguously good or bad? If it can be either, are there
| some questions I should check? (As a background, I am quite happy
| writing my queries in SQL)
| cj wrote:
| "It depends"
|
| There's nothing inherently wrong with Snowflake. But like any
| database it has a time and place.
| nattaylor wrote:
| In my experience it excels at the stated use case of OLAP in
| the cloud with compute separate from storage and claim it's
| unambiguously good for this.
|
| For OLTP it would be unambiguously bad (although maybe there's
| hope with Unistore, which I haven't tried.)
|
| Many workloads are a mix and so it can become ambiguous whether
| it's a great fit / great value / whatever you're defining
| good/bad-ness by
| mr_toad wrote:
| As a data warehouse it's great. It's not a complete solution
| for an analytics platform. Snowpark seems are bolted on and
| doesn't really benefit from the core engine. It's reporting
| capabilities are a bit weak. So you'll probably want some
| external BI tools and/or tools like R and Python if you're
| heavily into analytics.
| tiahura wrote:
| Any plans for cheaper plans with less or older data?
| xyst wrote:
| The "pay upfront" cost is $2,000/month. This is for data that was
| obtained via a network of shady data brokers. Visa, MC, banking
| industry can fuck off.
|
| Users likely not reimbursed for their data getting packaged and
| resold by merchants, payment networks, or credit card issuers
| hammock wrote:
| This data was already available to purchase long before this,
| for far more. Acxiom, Epsilon, etc
| dogman144 wrote:
| Getting downvoted for a true statement.
|
| Reading up on programmatic advertising from a perspective of
| learning it vs resources that critique it is illuminating.
|
| You're describing how it works. Use cash and buy offline with
| your cellphone left at home (yes, all 3), or get your data sold
| for other people's products.
|
| I used to be more vocal about privacy but reviewing the innards
| of the tech make me think this dynamic will never change. Too
| much money and too well built.
| TradingPlaces wrote:
| Reeks of selection bias.
___________________________________________________________________
(page generated 2023-11-01 23:00 UTC)