[HN Gopher] Launch HN: Jika (YC S20) - Price A/B Testing for Sho...
___________________________________________________________________
Launch HN: Jika (YC S20) - Price A/B Testing for Shopify
Hi all! I'm Kennan and I recently founded Jika (https://jika.ai).
Jika helps Shopify sellers A/B test their product pricing so they
can make more money. Let's say you sell shoes on Shopify for $100
and want to test $110. You'd 1-click install the Jika Shopify App
and go to a dashboard where you select and start a test for $110.
We then handle the heavy lifting of showing price A to 50% of your
visitors and price B to the other 50%. As the test is running, Jika
keeps track of revenue, conversions, and visitors at each price
point. You'd repeat this process as much as you can since the
optimal price for a product is always changing. Up until now, most
people have priced off competitors, but that doesn't cut it anymore
with big companies getting more and more sophisticated. A prime
example of this is Amazon accelerating their development of in-
house brands (~640 in late 2019) and using their dataset to
outprice other online sellers (2.5m price changes per day in 2013
and more today). One recent test of a best selling product for a
Shopify brand doing 1M+ monthly revenue resulted in a data
significant ~10% revenue lift from a -7% price. This better price
for the best selling product translates to a ~30k+ monthly revenue
increase (~180k+ revenue over 6 months). I chose to work on
pricing because I love optimization. Past optimization includes
dropping out of college after 1 year to be a growth (A/B testing)
engineer at Pinterest for 3 years and building an app for discounts
+ promotions at Hulu. In high school, I also optimized enough to be
a top 200 player in League of Legends. Would love to answer any
questions or comments below! PS. If you know anyone on Shopify,
I'd love to help them nail down their pricing. :D
Author : kennandavison
Score : 72 points
Date : 2020-07-16 17:34 UTC (5 hours ago)
| alphagrep12345 wrote:
| How does it compare to other A/B testing apps? Eg:
| https://apps.shopify.com/neat-ab
| kennandavison wrote:
| Neat A/B is a 100% daily traffic swap app. This approach isn't
| a valid a/b test since it isn't properly isolating the variable
| (price).
| neatab wrote:
| Neat A/B Testing here! First off, congratulations on the
| launch. I would argue showing different prices to
| simultaneous visitors is an ethical gray area - not to say
| it's not something that shops do, or wish to do. But that's
| why we choose to cycle pricing. In our experience, with
| proper consideration of cycle timing, test length and traffic
| consistency, it's possible to get solid results this way.
| newfeatureok wrote:
| If this were to become popular what would stop someone from
| building something to do the reverse of this, e.g. see if the
| item could be cheaper if you bought it as another customer/from
| another device/computer/location?
| kennandavison wrote:
| Yeah, someone could totally build Honey for Jika if they really
| wanted to here haha. If you think about it, discounts that
| don't apply to everyone (what Honey targets) are pretty much
| unorganized price a/b testing.
| asattarmd wrote:
| That's probably a good problem to have! Nonetheless, customers
| are still buying from the same store, so, it's a win-win.
| Breaking customer trust is probably the only thing that's bad
| here.
| kennandavison wrote:
| Yeah, it's a tradeoff for sure. I would say discounts that
| don't apply to everyone are a less organized way of a/b
| testing price. So a brand that does discounts all the time on
| certain cohorts of their users is going to have an easier
| time swapping over to using Jika vs. a brand that has 1
| price, no discounts ever, never have changed their price.
| lazydon wrote:
| May be it's just me, but the first thing reading that name
| reminds me of is the Zika virus. I wonder why would you choose a
| name for the startup that's so close to something with horribly
| negative connotations. Especially, in today's world scared to
| hell by viruses.
| kkt262 wrote:
| Why do you think you got accepted into YC? I'd love to check out
| your application.
| gleb wrote:
| Small suggestion: include how contribution margin changed in that
| example. That's the Northstar metric for a typical ecommerce
| company.
|
| Including it would sound more credible, and would prevent a
| typical customer from doing the math in their head of profit
| margin where this change becomes profitable.
| kennandavison wrote:
| Thanks for the suggestion! Contribution margin/profit is
| another metric we're going to track in the future for sure.
| dyeje wrote:
| Do most Shopify stores have enough traffic for this to produce
| real results? Seems like the vast majority of Shopify sites are
| small and are just going to get random results due to a small
| sample size.
| kennandavison wrote:
| Depends on the magnitude of the revenue increase.
|
| One thing that inspired me to build Jika were the stories of
| sellers doubling their price and actually increasing their
| number of conversions. I haven't had a customer like that yet,
| but I had one person who reduced their price 30% and increased
| their conversions 5x+. In that case, it's pretty clear there's
| a winner.
| dustin wrote:
| 30% might work in software, but it's is a very deep discount
| for most physical retailers.
|
| If the product has a 50% margin (sells for $100, costs $50 to
| source) then your retailer makes $100 for 5 orders instead of
| $50 for 1 order.
|
| Except they're probably keeping their own inventory, so they
| now have to keep $250 of inventory for that $100 in profit,
| rather than $100. This is problematic because a lot of online
| retailers are limited by their working capital.
|
| This is a drastically simplified example, but the bottom line
| is that physical retailers think very differently than
| software sellers.
|
| Great idea for an app either way. I wish you the best of luck
| with it as you learn the territory. For something similar in
| the Amazon space (a platform that lends itself much less to
| split testing) check out splitly.com
| break_the_bank wrote:
| Congratulations on the launch! Curious why you went with venture
| funding over indiehacking/bootstrapping?
|
| Heard this podcast a while ago[0] where they discussed main
| dishes(who new platforms) and side dishes(github apps / shopify
| apps). At the moment Jika looks like a side dish, which seems
| well served by the bootstrapping community.
|
| Edit: Add link to podcast
|
| [0]https://www.indiehackers.com/podcast/152-tyler-tringas-
| and-j...
| kennandavison wrote:
| Thanks for the congrats!
|
| Yeah, Jika probably would've been a profitable side project,
| but one of the goals here is to completely change pricing. It's
| super broken for most companies right now and needs to be fixed
| asap.
| jasoncui wrote:
| Awesome product and love the landing page. Super simple and
| clear.
| kennandavison wrote:
| Thanks Jason!
| hn_throwaway_99 wrote:
| I recall seeing this blog post a while back arguing you should
| never A/B test your pricing strategy (IIRC it was posted on HN,
| can't remember):
| https://www.priceintelligently.com/blog/bid/180676/why-you-s... .
| I'm curious about your thoughts on the points in this post.
| mritchie712 wrote:
| ProfitWell is targeted at B2B SaaS, very different from
| ecommerce (Shopify).
| kennandavison wrote:
| Re point 1: in many cases people actually DO have the sample
| size if the magnitude of the revenue increase is large enough
| (e.g. double price and increase conversions - lots of stories
| about this).
|
| Re point 2: in the end increased revenue from better pricing
| means you're capturing more of the value you're delivering to
| customers. but yes, there's a brand decision to be made here
| between 1 price that you always keep the same vs. a changing
| price that is more optimal.
|
| Re point 3: companies should always be testing their pricing
| since external factors don't stay the same.
|
| PS: this is also on the blog of a dynamic pricing company so
| they don't want the people on the fence about handing over
| pricing control thinking about a/b testing price haha.
| shafyy wrote:
| > In high school, I also optimized enough to be a top 200 player
| in League of Legends.
|
| Sorry but that's super obnoxious. No one in their right mind
| would use the verb "optimize" in this context.
|
| Feel free to downvote, but sometimes you need to call people on
| their bs.
| frakkingcylons wrote:
| That's a big achievement to be honest, so who cares about the
| word they happened to use there. Takes a lot of focus and work
| to be that good at strategic games like League, Dota, or
| Starcraft.
|
| Edit: Actually I think optimize is appropriate in that context
| too. To be that good, you have to min/max every game mechanic
| you can.
| GonzaloQuero wrote:
| I see the value on this. As a very, very bad player, but with
| some knowledge, it's a game that rewards heavily the ability to
| strategize, optimize builds for different, very quick
| situations, and shows dedication.
|
| Unfortunately, I agree that it's a poor thing to say in a
| presentation, as it requires people to actually know the game
| to understand why that would be good. People who don't play
| might just think "what do I care that he's good at a game?"
| tmpz22 wrote:
| This definitely originated from the Chris Sacca anecdote about
| Travis Kalanick being one of the top Wii Tennis players - it
| either came out in a start-up podcast or some shark tank
| episode.
|
| Anecdotally watching and following top e-sports players (in
| Overwatch in particular) I would see high video game
| performance as at best a neutral indicator... those archetypes
| of people are typically very good at the one activity they do
| all day and very poor at taking care of themselves or sticking
| to things that don't immediately excite them.
|
| Finally, for many video games are a coping mechanism and may be
| a better red flag indicator for past trauma, depression, etc.
| neolog wrote:
| I don't see the problem.
| greenbay20 wrote:
| This is dope. As close as one can be to the money, niched down
| and in a growing market. Good job! Keep us updated =D
| mtnGoat wrote:
| Can you easily opt traffic out of testing based on traffic
| source? for instance Google Shopping(Microsoft Shopping as well)
| is pretty strict about the price in the ad page, matching the
| price on the product page.
|
| do you plan to offer this product for other platforms? the number
| of Shopify merchants with traffic and sales large enough to get
| good significance in a timely manor is rather small, IME.
| thrownaway954 wrote:
| did anyone else do a double take and think the name was Jira?
| shafyy wrote:
| Nothing against you personally, but I'm the only one who thinks
| it's weird that a Shopify A/B testing app got into YC?
| mritchie712 wrote:
| they could have gotten in with something more ambitious, then
| pivoted to this when they realized it wasn't going to work. Not
| uncommon to launch something different than what you were
| accepted with (see Brex)
| shafyy wrote:
| Good point
| kennandavison wrote:
| This is what I applied with.
| greenbay20 wrote:
| I'm incredibly disappointed this was the top comment. Instead
| of asking whatever it is you want to know in an optimistic way,
| you just threw a random comment that with decent chance just
| makes the OP feel worse.
|
| For starters, a really simple question you could have asked
| that may update your belief of how weird it is they got into
| YC: "How much money are they making and how much are they
| growing per week?"
| shafyy wrote:
| I do think that an A/B testing platform can be a great
| business, and making any business work is for sure an
| achievement.
|
| I just had the impression that YC tries to back more
| "ambitious" or "important" businesses, but may be I'm wrong
| about that.
| kipply wrote:
| ambition or "importance" don't always manifest in the early
| stages of a startup (they probably usually don't). Shopify
| started as a snowboard selling company and I'm sure that
| the founders here pitched YC a larger vision that was
| appreciated.
| AVTizzle wrote:
| I appreciate the compassionate nature of your comment, but
| anecdotally I can very much attest to this thought being the
| first thing through my mind when I read the post title.
|
| This question sitting at the top further validates that other
| people are wondering it, so disappointing as it may be -
| people are curious!
| _xgw wrote:
| Maybe there's potential to port the app to other platforms!
| kennandavison wrote:
| Yep, going to do all online pricing.
| quadrature wrote:
| Thats what the implementation is today. the larger idea/problem
| is e-commerce optimization which is a big problem worth taking
| a bet on.
| extesy wrote:
| I'm actually surprised this is not already built into Shopify
| by default. This seems like a very basic functionality which
| can be added at any moment so what would that do with a
| startup?
| kipply wrote:
| Former Shopify employee, but speaking based on public
| knowledge;
|
| 1. Liability. A result of "raising prices by $10 is 5% more
| revenue" is not confirmation that raising prices by $10 is a
| good idea. I expect a non-trivial error rate, especially
| since most merchants aren't statistics savvy (forming
| experiments, doing risk analysis and interpreting data is
| hard). If Shopify causes this, it's a story. Startups will
| have room to develop features / legal to adjust to the
| liability concerns
|
| 2. Channels. Shopify is a very multi-channel platform and
| many stores actually rely on Facebook or Instagram
| integrations and physical stores rather than their online
| store. To be able to control prices on all these channels is
| an engineering challenge, and will also further dilute the
| value of the data. It'll also amplify concerns about pricing-
| inconsistencies.
|
| 3. Just a high-risk idea. Customers don't like seeing
| different prices. Merchants don't like running experiments
| that could cause them to lose money. Merchants also like
| being "good" to their customers by some measure, and some
| merchants would consider this unfair.
| lancesells wrote:
| I agree with all of this. If there was any store I shopped
| at and realized there were two different prices I would
| think they have some savvy marketers but would also stop
| being their customer. I don't want amazon or airline
| fluctuations in price when I'm shopping.
|
| Best of luck though. I can see many stores thinking this is
| the way to unlock profitability.
| break_the_bank wrote:
| Yep, that's the first thing that came to mind. Seems like a
| great bootstrap / indie venture but they raised VC money, I
| don't know why.
|
| The founder does say that they want to change all of pricing.
| Even then I'd argue that launching making recurring revenue
| would have got them more favourable terms with a VC in the
| future.
| pryelluw wrote:
| This might be their Apple 1 and not their iPhone.
| zatel wrote:
| This seems like a great shopify app I will certainly be adding to
| my next store. Do you have any go to optimization resources to
| share? (just general stuff on how to think about optimizing
| things in ones own life)
| aresant wrote:
| Handful of good links on price elasticity testing
|
| 1 - https://vwo.com/blog/ab-testing-price-testing/
|
| 2 - http://www.conversionvoodoo.com/blog/2012/12/why-price-
| elast...
|
| 3 - https://prisync.com/blog/price-testing-how-to-test-
| different...
| kennandavison wrote:
| Keep in mind price a/b testing is legal given that you aren't
| doing it off protected classes (the VWO post says that price
| a/b testing isn't legal).
| kennandavison wrote:
| Awesome, feel free to email me at kennan@jika.ai when you get
| started and I'll personally onboard you if you'd like! And in
| terms of optimizing, I think the biggest shift for me was
| keeping everything I want to do in a list (Google Doc ha) and
| building momentum by doing the easy things.
| purple_ferret wrote:
| Do privacy addons block this and if so, does your price tracking
| account for it?
| psadri wrote:
| I remember people used to hate finding out they were shown a
| different price from other people for the same product. Has this
| changed?
| neonate wrote:
| Could that problem be solved by changing the price at different
| times, rather than for different people in the same time
| interval? I wonder if that's why Amazon is making 2.5m price
| changes a day like the OP says above.
| kennandavison wrote:
| Unfortunately this wouldn't be a proper a/b test since it'd
| be comparing 1 day of 1 price to another day of 1 price. Only
| by splitting traffic 50/50 do you properly isolate the price
| variable.
| neonate wrote:
| Why can't the time slices be much shorter, say 15 minutes
| rather than a whole day?
| dividuum wrote:
| Most likely, although that solves a different pricing aspect.
| I would guess most of those are the result of automated
| pricing optimizations to influence your offers position among
| other sellers for the same product. There are services that
| automatically do this for amazon and other price comparison
| services. It can also result in a book priced at $23,698,655:
| http://www.michaeleisen.org/blog/?p=358 :-)
| kennandavison wrote:
| Haha, when your pricing algo has no ceiling/floor.
| kennandavison wrote:
| Addressed this a bit in another comment, but if you think about
| it, non-public/targeted discounts are pretty much unorganized
| price a/b testing. Also, the alternative is having sub-optimal
| pricing which isn't going to cut it pretty soon with Amazon (+
| other big companies) dynamic pricing tech + in-house brand
| expansion.
| misterbwong wrote:
| I'd argue there's a qualitative difference between A/B
| testing regular pricing vs. testing via targeted discounts.
|
| If your friend finds out you got a "private sale" price, this
| doesn't seem to violate the "fairness" principle because you
| were a part of something or got targeted for a _reason_. It
| might actually end up increasing the friends FOMO /jealousy
| and get them to find ways to join in on the discount via
| mailing lists, insider clubs, etc.
|
| A/B testing pricing might come off as arbitrary and not
| easily reasoned away thus violating a sense of fairness (at
| least during testing).
|
| I don't see this as a huge issue as A/B testing is usually
| limited time/scope but something to think about, I guess.
| pkiv wrote:
| Congrats on the launch! Have you thought about exposing the API
| instead of tightly coupling to Shopify? Similar to Split.io but
| for product pricing?
| itsdavemartin wrote:
| One observation: Pricing at $199, $499, $899 per month for a
| single feature seems interesting considering Shopify itself
| charges $29 $79, $299 per month.
| gustaf wrote:
| Considering the value of Jika I'm surprised it's not higher
| price.
| kipply wrote:
| Is there an AI(specifically deep learning) element to optimally
| distribute the A/B testing ratios or to suggest products/prices
| that may be good to A/B test? (Referencing the .ai url)
| pgodzin wrote:
| Few questions:
|
| 1. How do you determine who fits into the A cohort and who fits
| into the B cohort? If I refresh could I fit into a different
| cohort and thus see a different price? If I go Incognito?
|
| 2. Instead of a 50/50 A/B test, can you multi-arm bandit a
| handful of different price points and shift a higher proportion
| to the "best" one? And thus be able to change over time as well?
| kennandavison wrote:
| 1. Using random id + localstorage, so yes, incognito could
| result in a different price.
|
| 2. I'd love to implement MAB, but focused on nailing core
| product for now.
| kipply wrote:
| How do you avoid customers changing their behaviours by
| observing pricing changes? Or when they share it with a
| friend and go "cute shirt for 10 dollars" and their friend is
| like "wat it's 15 dollars"
| usaphp wrote:
| I just want to point out that I like a way you describe what your
| product does. Simple, clear examples and result. I like when
| people do that instead of asking for your email just to see what
| your product is even about.
| jasoncui wrote:
| +1 to this
| kennandavison wrote:
| Thanks! Keeping things simple + self-explanatory is a big focus
| with Jika.
| [deleted]
| akbar-saliev wrote:
| If this becomes popular what would stop Shopify from building
| this?
| kennandavison wrote:
| There's a lot of things Shopify should build, but yes, agreed,
| I am at their mercy in many ways. Going to diversify in the
| near future.
| dahdum wrote:
| I'd guess public outcry - people despise this type of practice
| and Shopify endorsing it would hurt their brand.
___________________________________________________________________
(page generated 2020-07-16 23:01 UTC)