[HN Gopher] A Google bot scrapes pricing info by adding items to...
___________________________________________________________________
A Google bot scrapes pricing info by adding items to carts
Author : psim1
Score : 185 points
Date : 2020-07-01 17:32 UTC (5 hours ago)
(HTM) web link (www.wsj.com)
(TXT) w3m dump (www.wsj.com)
| vmateixeira wrote:
| Genuine question, is this not considered a DoS attack?
|
| Let's imagine I have my online stock linked to limited physical
| items/assets, ex tickets for a show, which will get reserved for
| a period of time. This will be preventing genuine clients from
| buying them.
| nxpnsv wrote:
| Possibly it is lower traffic than a full on dos?
| vmateixeira wrote:
| Yes, in regards to traffic. But it's still denying me from
| providing a service to real customers.
| zelly wrote:
| You can always update your robots.txt or block the Googlebot
| UA. (lol)
| Mizza wrote:
| I'm thinking - if I forbid this in my site's Terms of Service,
| will DoJ go after Google for CFAA violations like they did to
| Aaron?
| vmateixeira wrote:
| Yeah.. probably depend$ on how _loud_ you can make yourself
| heard..
|
| RIP Aaron
| yongjik wrote:
| robots.txt, man, if you don't want search engines to visit
| certain part of your page, use robots.txt!
|
| Once heard a tale of an angry site owner calling Google (back
| when Google itself was novel) - Google deleted his whole website!
| Turned out he had "DELETE" button in each page, which generated
| plain GET request. So Googlebot visited the site, followed links
| to every page, and then of course followed every link that
| generated GET requests - because they are supposed to be safe.
|
| Don't be like that site owner.
| a1369209993 wrote:
| That has nothing to do with robots.txt, the problem is doing
| things in response to GET requests. I've said it before and
| I'll say it again: you do not _do_ things on GET requests.
| [deleted]
| YetAnotherNick wrote:
| How do I use robots.txt to tell google to not add item to the
| shopping cart?
| yongjik wrote:
| Erm... hide the shopping cart page behind robots.txt?
| rubyron wrote:
| Well, theoretically, your Add To Cart button could have an
| href with a path that's banned in robots.txt, but overridden
| with JS.
|
| But most online stores should be happy to have Google
| crawling their prices and showing up under the Shopping
| results.
| mlvljr wrote:
| That basta*d.
| bravoetch wrote:
| TL;DR - it's a google bot
| CraneWorm wrote:
| it was a google bot. But now the idea is out there to copy.
| amflare wrote:
| Specifically one trying to get the price at checkout
| whoisjuan wrote:
| This is the problem that I have with HN editorializing titles.
| This comment made perfect sense and was useful before they
| changed the title, but now that is changed it looks like the
| poster is an idiot who is just saying what the title says and
| some people downvote it.
|
| I know HN is not very keen on adding features, but this is one
| that is missing for the sake of transparency (seeing if the
| original title was editorialized)
|
| I understand that the original title was click-bait trash and
| this one makes sense, but it would be nice to understand how it
| changed so certain comments don't get de-contextualized.
|
| But I guess is the same problem with editing comments.
| caser wrote:
| This feels like a great way to get data on how all these
| different e-commerce companies approach remarketing.
| leoh wrote:
| Such a bot could be used to damage ad tracking
| rkagerer wrote:
| Are there legal implications to Google bots transacting with
| websites under false pretenses?
|
| I mean their normal web crawler identifies itself as such. Here,
| I feel like they're committing (very) minor fraud by putting in
| fake shopper information and actively hiding their identity. Not
| a big deal if it were just some Joe Schmoe somewhere, but at
| their scale might it border on harassment? The robot equivalent
| of a prank call?
| the_pwner224 wrote:
| Probably a violation of the CFAA. Lots of people hate it
| because they think it's overreaching, and lots of companies use
| it to legally threaten scrapers and security research. But in
| this case Google is doing mass unauthorized use of other
| people's computers.
| shadowgovt wrote:
| If I'm doing price comparison between online vendors, I will
| ---as a human---put some items in the cart and get right to
| the edge of checkout to determine what my final bill would
| be. I may not close the sale if I'm looking at a better
| option elsewhere.
|
| How is what I'm doing materially different from what Google's
| doing? Is scale a factor that matters for CFAA?
| jogjayr wrote:
| I think that's outdated information. ToS violations aren't
| prosecutable under CFAA since April.[1]
|
| 1. https://www.eff.org/deeplinks/2020/04/federal-judge-rules-
| it...
| inetknght wrote:
| You should worry more about sellers engaging in anti-
| competitive behavior like bait-and-switch or price fixing.
| whoisjuan wrote:
| This bot is simply trying to get the final price (with tax and
| shipping) which is ridiculous because e-commerce storefronts
| should do that in the first place without going through the whole
| checkout process.
|
| I always have found that kind of shady but it's probably known to
| increase conversions.
|
| What I found interesting is that this an open attack vector for
| e-commerces. Multiple bots can hit a website and start adding
| items and start the checkout process. This basically creates an
| unprecedented cart behavior data influx that ruins any possible
| usage for data coming from legit customers. Maybe cleaning the
| data wouldn't be that hard but if someone knows what they are
| doing they can really make it hard (separate IPs, emails and cart
| behavior)
|
| I doubt Shopify or Magento have anything to prevent this.
| lalos wrote:
| Also, seems like a clever hack for automated scraping after all
| most carts are pretty uniform in their structure.
| mmcconnell1618 wrote:
| Not all shipping charges can be calculated ahead of time. For
| example, you may offer free shipping on orders over $50. You
| may charge $9.99 for the first item, $5.99 for each additional
| item. You may charge by weight of the whole order. You may have
| oversized items or packages that can be combined to reduce
| shipping charges. Some items may ship together as OTR Freight,
| while others can go via the local postal service. Buying
| multiple items changes this calculation.
|
| So, yes, you can estimate shipping for a single item but you
| can't always present the per-item shipping charge as it depends
| on the context of the whole order.
| jasonv wrote:
| What this poster said.
|
| Yes, a lot of smaller e-commerce platforms could do this, but
| finalizing order value can be a very complex workflow for
| bigger merchants with more varied sku mixes.
|
| I've worked in multi-billion dollar Ecom companies where the
| programs to refine the order checkout process gets scoped as
| a multi year effort accounting for a couple of decades of
| legacy cruft... even if you separate the
| "product/tax/shipping" calculations from the
| "customer/credit/rewards" dependencies. But it's often not
| worth separating them because they're very inter-dependent.
| Moreso when you involve drop shipping or made-to-order
| things.
| bluntfang wrote:
| Well we can change this by including shipping as the total
| price and not give deals on shipping. Deals on shipping are
| dark patterns.
| chrisan wrote:
| How does that change by having the bot add items to the cart?
| You haven't solved anything
|
| You are still left with the same scenario as if the store
| listed the individual shipping price on the front page
|
| Google isn't going to know what other items you _might_ add
| to show you a "real" shipping cost
| Xelbair wrote:
| And what's even more interesting - human would do exactly
| the same thing.
|
| Add items to card, check the total price and then decide
| whether to buy it(i remember trying to order some stuff
| form one japanese plamo store - and it didn't provide exact
| prices before checkout. I went through the process, but
| even the cheapest option for delivery was way too high - as
| 2x price of the whole order)
| hrktb wrote:
| I'd assume parent's point is regarding the "which is
| ridiculous because e-commerce storefronts should do that in
| the first place without going through the whole checkout
| process." part.
|
| There's a lot of legitimate case were showing shipping
| price upfront is just not doable or valuable to the
| customer.
|
| BTW there are a surprising amount of shops for specialized
| goods that won't even list the final price at the end. The
| customer places an order, and they update it with a
| finalized price after a human looks at the content, and
| from there the customer is free to pay the transaction or
| give up the order.
| zoomablemind wrote:
| Even the Y2K-style ecommerce stores usually had a
| separate S&H section for some guidance. These days the H
| part (handling) seems less in vogue (perhaps still common
| on ebay), while S part is pretty predictable if not free.
|
| It's the T (taxes) part that may be still a tipping point
| these days, but it's just between vendor and your state,
| saltedonion wrote:
| It matter because if you purchase multiple things your
| average shipping cost per item changes. If you only
| calculate shipping based on first item shipping cost it
| will be inaccurate.
| mdoms wrote:
| But Google is only showing a result for a single item.
| clairity wrote:
| that's true that the calculations can get complicated pretty
| quickly in ecommerce, but google probably has all the data it
| needs (origin zip, destination zip, likely carrier(s),
| possibly even the weight/size of each item) to provide a
| pretty good estimate in most cases. they could even calculate
| a range for [1 item per box, all items in 1 box].
|
| the important bit is to present it as a separate line item
| (with grand total) so that consumers can decide how much to
| trust the estimate.
|
| that would be an even clearer shot across the bow of amazon,
| walmart, and the like, who provide comparison across their
| own platform merchants, but not across all merchants
| everywhere.
| gwright wrote:
| Google guessing seems like a terrible idea. That will just
| confuse the consumer when they go to purchase and find a
| different value, possibly creating a customer service
| problem for the vendor through no fault of their own.
| gowld wrote:
| There's a tiktok meme doing this to harass the Trump campaign's
| online store.
| sp332 wrote:
| _e-commerce storefronts should do that in the first place
| without going through the whole checkout process_
|
| Yes but how would you verify this or hold them accountable?
| gruez wrote:
| > This bot is simply trying to get the final price (with tax
| and shipping) which is ridiculous because e-commerce
| storefronts should do that in the first place without going
| through the whole checkout process.
|
| It's usually not possible because you don't know how much the
| shipping + taxes are until the customer enters the billing
| information.
| stronglikedan wrote:
| I just used a website that had a simple form at the bottom of
| the cart. It had one text input for the postal code, and a
| button to get the rates to that postal code based on what was
| in my cart. IMHO, this is how it's done right, since all you
| need to know is general location and weight.
| mkl wrote:
| Is that US only? You need country too.
| rietta wrote:
| For commercial scale products you need more than that.
| The full address. Is the address residential vs
| commercial with a loading dock? That and more factors
| impact the shipping price a lot! Logistics companies have
| people who have to research an address and look at Google
| Earth photos of the property to answer these questions.
| jiofih wrote:
| Maybe I haven't done enough online shopping recently, but
| as far back as I remember this used to be the norm: enter
| postcode to calculate shipping, get precise final price
| without even adding to cart. Is it not the case anymore?
| the_pwner224 wrote:
| On many websites it still is. But recently many smaller
| independent stores use the Shopify platform where
| shipping is the penultimate step, before billing. You
| have to give address, email(!), phone number(!,
| mandatory), etc. before getting the price. I normally
| just use a fake email and number to get shipping price,
| and then do the actual checkout in incognito. Pretty sure
| if you do enter your real info and don't continue with
| the purchase then you'll get email spam telling you to
| buy stuff.
| [deleted]
| jimktrains2 wrote:
| Phone number is required or otherwise highly encouraged
| by some shippers, like FedEx.
| gruez wrote:
| I'm pretty sure most sites (eg. Amazon) does the same
| thing. Probably for the reason you mentioned: so they can
| have your contact information to send you spam later.
| whoisjuan wrote:
| Yeah that's true. But from a UX perspective there are ways to
| make this less opaque. Perhaps a call to action at the top of
| the listing with an entry box to enter the Zip Code so an
| approximate final price can be calculated.
|
| Any good UX designer can come up with a solution for this in
| a couple of hours or less. There's just no motivation to make
| it happen because this obfuscation of data is particularly
| optimized and useful for the sellers.
| neutronicus wrote:
| e-commerce sites seem to be asking for my location all the
| time anyways.
|
| If they're doing that anyways, they should have everything
| they need to hazard a pretty good guess (and then they have
| an actual inducement for me to provide it).
| whoisjuan wrote:
| Agree!
| jeanvalmarc wrote:
| Amazon just lists it as a "subtotal," which I think is
| probably the best way to do it. I don't know about the "UX
| designer in a couple of hours" line: automatic shipping is
| a nightmarish bag of worms and isn't really a UX problem.
| What do you do if they order multiple SKU's that don't pack
| nicely into one box or are warehoused in different
| locations? Or if there is something weird about their
| shipping address? What do you use for box size and weight,
| and what approximation did you use for dunnage?
|
| You can approximate UPS/FedEX costs by fitting a trend
| line, they are decently modeled by a linear (base charge +
| K*distance) function, but when you go to buy the label you
| might be way off. This puts you in the lose-lose-lose of
| either eating the difference of wrong estimates,
| overcharging for shipping and losing conversions, or just
| making people hate you by increasing the shipping over the
| estimate. Making a shipping API call is noticeably slow, so
| most people require an interaction after the address is
| entered.
|
| Tim Sweeney's hot take was that "the two hardest problems
| in computer science are cache invalidation and shopping
| carts!"
| TeMPOraL wrote:
| ZIP code box that doesn't oblige you to provide any more
| accurate data, and also without it, it should still be
| possible to display _brackets_. "Shipping: $6 - $24,
| [enter ZIP code for detailed quote]".
| wdb wrote:
| They could at least include the tax in the price. That's
| normally fixed depending on the item category e.g. low or
| high tax rate. Only the US is being weird with its taxes.
| ww520 wrote:
| Tax is depending on where the buyer is coming from.
| Ruthalas wrote:
| I will bail from the purchasing process on sites that are
| unwilling to give me a final price before I enter payment
| information.
|
| A postal code should suffice, and I'm not providing more
| personal information if the site is unwilling to say what
| I'll be charged upfront.
| texasbigdata wrote:
| Isn't that most of them? Unless you assume they eat
| variable shipping by 'fronting' you a fixed price.
| Apologies if I'm misunderstanding what you're saying.
| Ruthalas wrote:
| It is still typical (for which I am thankful), but that
| seems to be shifting a bit.
|
| A relatively common example would be shops using the
| Shopify platform.
| enriquto wrote:
| > It's usually not possible because you don't know how much
| the shipping + taxes are until the customer enters the
| billing information.
|
| Sure, they show you a different content depending on your IP
| address and lots of shady heuristics, but when it comes to
| estimate a shipping cost, it is absolutely impossible: you
| can just be anywhere, who knows where you are. I say
| bullshit.
| bronzeage wrote:
| Makes you wonder whether the smart thing to do is just make it
| convenient for the bots to get the info out so they won't ruin
| your data and waste your bandwidth. Can't fight them, join
| them. Can't really stop free information flow.
| stingraycharles wrote:
| They already do that, there's all kinds of XML and JSON and
| whatnot standards to communicate product info, inventory,
| whatnot. The reason Google is doing this is because this
| information cannot be trusted all the time, there will always
| be bad actors.
|
| The process may eventually evolve in a cat-and-mouse game,
| where malicious e-commerce sites try to detect these Google
| crawlers and serve different price info to them, but let's
| hope it doesn't get this far.
| viraptor wrote:
| Given the possibility you get detected and it impacts your
| organic search ranking... I'm not sure any serious vendor
| would risk it. And if they do, let them burn.
| WWLink wrote:
| Or they could stop spying on their customers and trying to
| figure out how to add dark patterns to maximize "engagement"
| and "conversion" lol.
|
| I mean yeah, I get that if you inadvertently make the
| checkout button hard to find, you'll lose potential sales,
| but I don't think you need intricate data about what your
| customers are doing to figure that out.
| kqr wrote:
| Or quietly detect the bots and feed them junk data after
| they've gone through the hoops. Not saying it's the better
| option, but knowing the business maybe the more likely one.
| inetknght wrote:
| > _Or quietly detect the bots and feed them junk data_
|
| Just a few customers with alternative browsers being
| detected as "bots" will poison your reputation and income
| stream.
| Spivak wrote:
| As someone who uses an alternative browser... I kinda
| doubt it. As a group we wouldn't even move the analytics
| needle let alone revenue.
| rabanne wrote:
| No, it definitely won't. In fact our card payment system
| has been updated to think alternative os and browsers are
| high-risk and even decline payment. We didn't have any
| revenue loss.
| toomuchtodo wrote:
| Which is why it should be a chrome extension (like Honey);
| exfiltrate the data out while providing the end user
| financial benefits. Messing with the data breaks the user
| experience and impacts revenue of the target site.
| crazygringo wrote:
| Not only that, but certain brands on certain sites _won 't show
| the price_, with that message "add to cart to see the price!"
|
| I've heard varying explanations as to why, but at the end of
| the day it doesn't matter. Adding to the cart is the _only_ way
| to scrape the price.
| cglace wrote:
| How do you show a final price if you don't have all the
| information needed: tax locality, shipping preference, total
| cart value discounts etc.
| alpha_squared wrote:
| If I'm remembering right, Best Buy used to have "deals" on
| items that they "couldn't show you" until item was in cart.
| They may still be doing this. Best Buy's justification for it
| was that its agreements with manufacturers prevented it from
| displaying items below certain prices on their site. I'd
| never seen this elsewhere to know how pervasive these
| agreements were (or if Best Buy was just taking losses on
| certain items).
| hundchenkatze wrote:
| Newegg still does this. Here's an item from their 4th of
| July sale.
|
| https://www.newegg.com/p/2AM-008Y-00003?Item=9SIAEG2BMZ4393
| Spivak wrote:
| KitchenAid is famous for these kinds of agreements as a
| measure to ensure the public perception of their value
| doesn't go down whenever there are sales.
| thaumasiotes wrote:
| > Best Buy used to have "deals" on items that they
| "couldn't show you" until item was in cart. They may still
| be doing this. Best Buy's justification for it was that its
| agreements with manufacturers prevented it from displaying
| items below certain prices on their site. I'd never seen
| this elsewhere to know how pervasive these agreements were
|
| This was also pretty common on Amazon.
| whoisjuan wrote:
| They still do it.
| miohtama wrote:
| All EU prices must be tax included.
|
| All hospitality prices must include cleaning and service fees.
|
| It is only the US were hidden fees and charges may apply and
| the price regulation, if any, is more tilted towards
| corporations.
|
| Only upfront shipping charge is tricky because it depends on so
| many factors.
| shultays wrote:
| I find not including tax better tbh. That way people are
| being reminded how much tax they are with every purchase.
| bcrosby95 wrote:
| The problem with the US is taxes are complicated to
| calculate. You basically have to geolocate an address because
| governments can add extra sales tax to arbitrarily defined
| regions. My friend lives in a particularly special area, and
| not even major retailers like Macy's gets his sales tax
| right.
| mkl wrote:
| It seems like it will also mess up item availability
| information. If an item has limited stock, bots adding it to
| carts could make it appear out of stock to real customers.
| alasdair_ wrote:
| >which is ridiculous because e-commerce storefronts should do
| that in the first place without going through the whole
| checkout process.
|
| On top of the other reasons mentioned, the seller may have a
| contract with the manufacturer that covers a "minimum
| advertised price"
|
| "The FTC says that the price displayed in a secure or encrypted
| shopping cart isn't subject to MAP because it's technically not
| advertising."
|
| (https://www.thebalancesmb.com/what-is-minimum-advertised-
| pri...)
| abofh wrote:
| Google.
|
| Saved you a click.
| lawnchair_larry wrote:
| Thanks. "A Google bot scrapes pricing info by adding items to
| carts" could have replaced that whole fairy tale that they
| wanted us to pay for.
| dang wrote:
| Ok, we'll use yours. Thanks!
|
| I kind of liked the mystery shopper angle, but since there's
| more than one complaint in the thread, the guidelines win ("
| _Please use the original title, unless it is misleading or
| linkbait_ ")
|
| https://news.ycombinator.com/newsguidelines.html
| justinwp wrote:
| Protip: You will often get a discount coupon if you go through
| most of the checkout process(need to provide email), but wait a
| couple days. Many stores automate abandoned checkout promotions.
| bradlys wrote:
| Yes! This is also something that is common with smaller online
| retailers. Don't expect this with B&H, Adorama, or Newegg.
| Frequently these small companies give one time codes you won't
| see or be able to gain elsewhere.
| webmaven wrote:
| For a while there were registrars that gave a discount when you
| abandoned your cart.
| Alupis wrote:
| The real problem with this is from the merchant side of things.
|
| This bot generates thousands of "Abandoned Carts" on one of our
| sites... thousands...
|
| We send cart reminders to Abandoned Carts after a few days,
| sometimes with a coupon offer to complete checkout.
|
| This bot is responsible for thousands of bounced emails each
| week, which impacts our metrics with Mandrill among other things.
|
| Maybe we shouldn't care, but it's sloppy and ruins all sorts of
| stats we keep track of regarding cart abandonment rates,
| recapture rates and more.
| px1999 wrote:
| Do your users consent to contact before you send them reminders
| or coupons? If not, you've earned your bounce rate.
| Alupis wrote:
| Of course we have consent. Not sure what kind of question
| that is?
|
| Violate CAN-SPAM Act and risk a $16,000 fine per instance? No
| legit business is going to do that.
|
| Just ask Papa Johns how painful those fines/settlements can
| be[1].
|
| [1] https://topclassactions.com/lawsuit-settlements/lawsuit-
| news...
| cortesoft wrote:
| I personally find the 'email abandoned carts' behavior to be a
| dark pattern
| Alupis wrote:
| Dark pattern or not, it's super effective - particularly when
| accompanied with a coupon ;)
|
| Besides, the user is opting-in to receiving these emails.
| They don't have to provide an email address - so some are
| probably playing the game and seeing if they get a coupon or
| not.
|
| As an aside - if the internet worked the way SV hipster
| brogrammers thought it should work, nobody would use it.
|
| Yes, ads are crazy effective - you can ignore or block them,
| we don't care because enough people don't block them and are
| happy to click.
|
| Yes, emails are crazy effective - you can ignore or opt-out
| or never opt-in, we don't care, you just cost us money if
| you're not engaged anyway so we'd rather you not be on our
| mailing list.
| cortesoft wrote:
| So you are saying these bot accounts have opted in to
| receiving emails? You don't validate the email when someone
| signs up?
| Alupis wrote:
| The site terms and conditions are displayed very
| publicaly and accessible to anyone who cares to read
| them.
|
| By entering your email address you are opting-in for
| transaction related emails, including new order
| confirmations, shipment notifications, and yes cart
| reminders. It's spelled out for you.
|
| It's the same for almost every ecommerce site.
|
| That's different than marketing emails, which require a
| separate explicit opt-in - ie. the user has to go and
| type their email address into another form and click
| "Sign up".
|
| It doesn't get any more transparent than that.
|
| Don't call something a dark pattern just because you
| can't be bothered to understand what you're consenting to
| when using someone else's website and start entering
| information like your email address or more. That's
| entirely on you.
| cortesoft wrote:
| I am not sure what your conception of a dark pattern is,
| but the idea that adding something to a TOS means it
| can't be a dark pattern is simply false.
|
| The whole concept of a dark pattern is about UX choices
| that lead people to agree to things that they don't
| actually want; it isn't about whether you break your TOS
| or not.
|
| I am saying, I think that if you asked people point blank
| "do you want websites to email you reminders if you leave
| something in the cart?", most people (myself included)
| would say, "no, I don't want to get that email"
|
| You can put whatever you want in the TOS, but it doesn't
| mean users like it, and a user agreeing to something
| doesn't mean they like all the things they are agreeing
| to.
|
| Whether it is "on me" or not, it is still a dark pattern.
|
| Also, as a user entering my email, I am not promising you
| that I will always accept email to that address from you.
| If you try to send me email and I reject it, that is 'on
| you' to deal with what that rejection does to your spam
| scores.
| malux85 wrote:
| Can't you useragent sniff the bot and cut it off?
|
| If u want help coding this or advice happy to help (for free)
| Alupis wrote:
| If you cut it off, you get penalized in Google Merchant
| Tools, and possibly have your product feed suppressed, which
| will dramatically impact your search visibility for both text
| and product searches. It can also impact your Google Ads if
| you link that with your product feed, and more.
|
| So, effectively, no you cannot cut this bot off.
|
| To make it worse, the bot doesn't always follow the same
| pattern. Sometimes slightly different names, addresses, etc.
|
| We initially thought it was fraud attempts, but none of them
| actually attempt a checkout. They just enter all their info
| on the checkout page, get the final quote, and bail.
|
| It would have been nice if Google told people about this
| instead of it just happening. Or allowed you to schedule a
| time slot for it to do what it's going to do.
| dzmien wrote:
| I wonder why Google didn't include a "cleanup" routine that
| empties the shopping cart after the data is collected. It
| seems like it would be a trivial thing to do, unless I am
| missing something. I guess the answer is because it would
| not benefit them in any way.
| Alupis wrote:
| We'd still have an abandoned cart, since the session was
| created and held some sort of data - but it would be far
| less disruptive for sure.
|
| Empty Abandoned Carts are useless anyway (for stats and
| other things - we track bounce rates in other ways), so
| that would be a large improvement from what is going on
| right now.
| Felk wrote:
| It sounds like you interpreted malux85 suggestion as
| cutting the bot off from putting things in the cart
| altogether. What I understood is that, given it's
| recognizable by user agent, those carts can be marked as
| created by a bot to exclude it from statistics and reminder
| mails only.
| Alupis wrote:
| Perhaps. Not everyone is lucky enough to have built their
| own ecommerce platform, so many people are at the whims
| of whatever tools Shopify, BigCommerce, 3dCart or others
| provide.
|
| For this particular problem, none of those platforms can
| provide any assistance.
| malux85 wrote:
| Good lord, that sucks, hopefully a google engineer is in
| this thread...
| SquareWheel wrote:
| >We send cart reminders to Abandoned Carts after a few days,
| sometimes with a coupon offer to complete checkout.
|
| I consider this spammy behaviour, and mark the emails as such.
| I can only hope this discourages such practices in the future.
| Alupis wrote:
| It doesn't. If you mark it as Spam through most email
| programs, it's reported to the sender (Mandrill in our case)
| and Mandrill automatically black-lists your email address so
| we don't continue to send to someone that doesn't want the
| emails.
|
| That's a win-win.
| matchbok wrote:
| Still an annoying and anti-consumer practice. Another
| "growth marketing" tactic that doesn't take into account
| the number of people who never visit that site again
| because of the spammy stuff.
| soganess wrote:
| For people saying this to calculate the final price with shipping
| and tax, it's not (or at least not entirely). It is for this new
| sales conversion dark pattern where prices aren't listed until
| you add to cart.
|
| Ebay sellers are particularly bad offenders:
| https://www.ebay.com/itm/Open-Box-Certified-Samsung-Galaxy-1...
| an_opabinia wrote:
| That clearly has a see details button that shows the price.
| soganess wrote:
| Most dark patterns have a non-intuitive way of circumventing
| them (the small-font faded-color "no, thank you" button comes
| to mind). That is Ebay's.
|
| Other examples here:
| https://ux.stackexchange.com/questions/83050/price-too-
| low-t...
|
| Amazon example from a few years ago: https://lh5.googleuserco
| ntent.com/ztyT6xTPaTr9TtP8LwlRJBE6RV...
| nkozyra wrote:
| What am I missing on here? That item has the price listed
| without having to Add To Cart.
| Felk wrote:
| The modal that pops up is not in the dom until you click the
| "See details" link, which has target="javascript:;". The "Add
| to cart" button is an actual link. I wouldn't be surprised if
| Google just doesn't want to run javascript to extract pricing
| information if it doesn't necessarily have to.
| Youden wrote:
| When and why did news cease being news and start being short
| stories and opinion? This entire article could have been cut down
| to the last few paragraphs and nothing of value would have been
| lost.
|
| Look at The New York Times in 1921 [0]. Generally the stories are
| factual and to the point. The entire front page seems to be pure
| news. There's very little storytelling here, at most there are a
| few timelines of events.
|
| Look at The New York Times today [1]. There's a bunch of factual
| and useful Coronavirus information but ~15% of the page is
| dedicated to "Opinion", the second article appears to be pure
| speculation, the third article is a bunch of storytime fluff
| around a little bit of news and the front page has a mix of
| actual news and opinion pieces being passed off as news.
|
| When did this happen? Why? Did people lose interest in actual
| news? Is there less actual news to report?
|
| Perhaps this is regional? Take for example the story about the
| San Quentin prison. NYTimes [2] has the same drawn out nonsense
| as this Google story while Aljazeera [3] adds a lot of background
| but sticks to factual reporting.
|
| [0]: https://archive.org/details/NYTimes_jul16_31_1921
|
| [1]: http://archive.is/oiiXU
|
| [2]: https://www.nytimes.com/2020/06/30/us/san-quentin-prison-
| cor...
|
| [3]: https://www.aljazeera.com/news/2020/07/san-quentin-prison-
| se...
| galacticaactual wrote:
| I view this phenomenon as a sort of sociological entropy.
| Anything bad that can happen will happen if the people tolerate
| it. Same with shitty politicians, encroaching of rights, etc.
| Its all just a matter of time.
| gretch wrote:
| This theory doesn't hold up against the test of time.
|
| We used to have tyrant monarchs - Genghis Khan would roam the
| steppe and cutting people's heads off and that would be
| normal every day life.
|
| There are still bad things today, but more people have more
| rights and a higher quality of life.
| e15ctr0n wrote:
| > _Genghis Khan_
|
| Genghis Khan was neither a tyrant nor a monarch, nor was
| cutting off heads a part of his everyday life. He actually
| got more Mongol people more rights and a higher quality of
| life.
|
| I would request you to read a detailed and accurate account
| of his life by a historian.
|
| For example, _Genghis Khan: His Conquests, His Empire, His
| Legacy_ by Frank McLynn https://amzn.com/B00X2ZW5ZI
| deno wrote:
| No to take away from your point, which I think is valid, but
| the NYT in 1921 is _the_ Internet. We've gone from being
| starved for information to being overloaded. So really you're
| comparing apples to oranges they just happen to be named the
| same thing.
| Aloha wrote:
| Opinion masquerading as news sells, see Fox News, MSNBC, et al.
| pgbi wrote:
| Because pure news are pretty much free today. So newspaper need
| to provide something else: analysis, opinions, etc. to be able
| to sell something.
| kulahan wrote:
| Not to mention we have metrics which tell us exactly what
| kind of reporting is most likely to have good reader
| engagement (and thus higher ad payouts). Most news sources
| are just giving us exactly what the majority of people want.
| nurettin wrote:
| Even HN does this. It is the magic of the reply button. As
| a result, they pull more screen time and have a stickier
| crowd to show "we are hiring" ads.
| ardy42 wrote:
| > When The Wall Street Journal contacted Google in June, a
| spokesman at the internet giant, after a few days of digging,
| provided an update: The mystery shopper is a bot of its own
| creation.
|
| > The purpose: making sure the all-in price for the product,
| including tax and shipping, matches the listing on its Google
| Shopping platform or in advertisements. It wasn't to cause angst
| to merchants due to thousands of abandoned carts.
|
| > "We use automated systems to ensure consumers are getting
| accurate pricing information from our merchants," a company
| spokesman said. "This sometimes leads to merchants seeing
| abandoned carts as a result of our system testing whether the
| price displayed matches the price at checkout."
|
| You'd think they could have better identified themselves in
| accounts they were creating rather than creating this mysterious
| "John Smith" persona. Maybe "GoogleBot PriceVerifier" would have
| been a better choice.
|
| edit: remove my inaccurate confusion about something, and fix
| quotes that I'd copied from a plagiarized version of the article.
| teh_klev wrote:
| > It reads like it was written by someone who hasn't fully
| mastered English
|
| English is my first language, I didn't find the writing that
| terrible. It's certainly no worse than the standard of output
| from most large news outlets. Could you cite some examples?
| ardy42 wrote:
| > English is my first language, I didn't find the writing
| that terrible. It's certainly no worse than the standard of
| output from most large news outlets. Could you cite some
| examples?
|
| Sorry, it was my mistake. Before I saw the archive link, I
| had found a reposting on some sketchy website and got the
| tabs confused. That version is basically plagiarized but
| makes some weird word substitutions.
|
| WSJ:
|
| > For more than a year, online merchants selling items
| ranging from kayaks to keychains have puzzled over the
| mystery shopper with the generic name behind thousands of
| abandoned carts. Each cart has only one item.
|
| https://apkmetro.com/who-is-the-mystery-shopper-leaving-
| behi...:
|
| > For greater than a 12 months, on-line retailers promoting
| objects starting from kayaks to keychains have puzzled over
| the thriller shopper with the generic title behind hundreds
| of deserted carts. Every cart has just one merchandise.
|
| WSJ:
|
| > The purpose: making sure the all-in price for the product,
| including tax and shipping, matches the listing on its Google
| Shopping platform or in advertisements. It wasn't to cause
| angst to merchants due to thousands of abandoned carts.
|
| apkmetro plagiarization:
|
| > The aim: ensuring the all-in worth for the product,
| together with tax and transport, matches the itemizing on its
| Google Purchasing platform or in commercials. It wasn't to
| trigger angst to retailers as a consequence of hundreds of
| deserted carts.
| hckr_news wrote:
| Strange. I think that second article was computer
| generated. I've seen programs that allow you to basically
| re-write an article or essay by running it through a
| program that replaces certain words with synonyms, and re-
| writes other parts of the essay. This person also has ads
| on their website and seems to be generating money through
| copying articles.
| teh_klev wrote:
| Ah, ok, that would explain things. :)
| [deleted]
| bluGill wrote:
| They need to be non traceable. If I'm doing something
| underhanded with pricing information I want to detect Google
| and other such bots and give them different information.
| inetknght wrote:
| You really think it's wise to lie to your customers?
| eatingCake wrote:
| If you're a medium sized corporation that is subject to the
| whims of individual customers, no. If you are a gigantic
| global company that can position itself to weather any
| temporary inconvenience, absolutely yes.
| shadowgovt wrote:
| I mean... The vast bulk of digital capitalism is
| information asymmetries. Lies by omission are nearly the
| currency of e-commerce.
|
| Is it wise to intentionally refrain from offering the same
| price to a human you'd offer to the Google-bot? Possibly.
| Depends on your goals.
| its_dario wrote:
| No, and that's not their point.
|
| They're saying if they were to lie to their customers,
| they'd want to make sure they're deceiving Google. In that
| case, having an easy way to detect that it's Google would
| make that trivial.
| Animats wrote:
| Now even the WSJ has clickbait titles. Should have been "Google
| price-checking system annoying merchants".
| hyperrail wrote:
| This is an A Hed, one of the Wall Street Journal's daily funny
| news stories on the front page. Other recent ones include:
|
| * Baseball Stadiums Are Closed to Fans - but This Guy's Balcony
| Is Open for Business
|
| * Americans Craving Contact Ponder New Rules for Throwing a
| Party in Real Life
|
| * When Your Best Friend in Quarantine Is a Squirrel, You May Be
| Going Nuts
|
| * Beware of Falling Tofu: China Takes on High-Altitude
| Littering
|
| * Did You Forget Things During Lockdown? So Did People With
| Superior Memories
|
| In that context I don't have a problem with the title "Who Is
| the Mystery Shopper Leaving Behind Thousands of Online Shopping
| Carts?".
| agustif wrote:
| Hahaha That's the NYT website headlines nowadays LOL
| baybal2 wrote:
| Funny, a one quick gig I did in my college years was to write a
| shopping bot protection against "guaranteed lowest price" scraper
| like tigerdirect, or RFD.
|
| Back then, the goal was exactly the opposite.
| danimal88 wrote:
| It's just price data collection. In particular, MAP policies can
| be skirted by not publishing a final price but having a price
| below MAP in the cart which is a common tactic that online
| sellers utilize. By pretending to walk through the cart, all
| sorts of data about pricing, taxes, etc. can be learned. It's not
| entirely uncommon to see different prices at different times, for
| different user agents, for different locations, etc. Used to work
| for a company that build huge price collection systems and built
| many of them...
| Drdrdrq wrote:
| MAP == Minimum Advertised Price
| tacon wrote:
| Would it be too much for Google to program the bot to get the
| final price, and then delete all the items from the cart? Seems
| rather rude, even for Google.
| disposekinetics wrote:
| Is abandoning a cart really rude behavior? I sometimes do it
| just to see if they'll spam me as a test of if I want to do
| business with a site.
| jawns wrote:
| It's not rude at a consumer level, where (in general) you're
| at least considering making the purchase. It's arguably rude
| at a bot level, depending on the frequency, where there is 0%
| chance of conversion.
| dragonwriter wrote:
| The entire purpose of the bot is to provide listings to
| consumers who are looking to buy.
|
| If it was consumer journalist doing it to get the price for
| a news article (in a for-profit publication) about the
| product, would it be "rude"? If not, how is it for Google
| bot?
| jsnell wrote:
| Because bots will do it at a much larger scale than
| individual humans. The first law of web robotics applies
| here: the bot should not harm the website it's crawling,
| or through inaction allow it to come to harm.
|
| I didn't read the article due to the paywall, but I
| assume that the problem is that the problem is that these
| goods are reserved for that (non)-customer until the
| shopping cart times out? That is directly costing the
| merchant money, either in lost sales or having to
| maintain extra inventory.
|
| So yeah, that bot really should have been programmed to
| end the session with an empty basket one way or another.
| rubyron wrote:
| An abandoned cart reminder email sent a few hours later has a
| ridiculously high conversion rate - around 15% in my
| experience. Online vendors aren't going to stop that
| practice, especially when the big e-commerce platforms make
| it easy to do.
| klyrs wrote:
| I abandon carts more often than not. Pretty much for the same
| reason as the bot: I wanna know how much I'm actually getting
| charged with taxes, shipping and coupons. I'll do similar
| orders on multiple stores, and only finalize the best deal if
| I'm satisfied with it. Sometimes I just "walk away" because
| nobody's selling at my pricepoint.
|
| Is this rude? I really don't care.
| s1k3s wrote:
| Is this supposed to intrigue me? Good bot
| vmception wrote:
| That sparked a funny idea in my head, what if we tricked product
| managers industry wide to follow KPIs and A/B tests that resulted
| in a better user experience for consumers, instead of experiences
| that coincidentally slightly upticked "engagement".
|
| Because it seems like this mystery shopper is already doing that.
| itronitron wrote:
| Given that engagement metrics have been heavily interfered with
| for many years, as a result of bots and other activities, and
| yet PMs still rely on them it seems unlikely that they will be
| pulled away from that spectacle anytime soon. I like your idea
| though.
| hundchenkatze wrote:
| > "and yet PMs still rely on them it seems unlikely that they
| will be pulled away from that spectacle anytime soon."
|
| I think they meant, since PMs will never stop using metrics,
| we should write bots that skew those metrics in favor of an
| experience for the consumer rather than the perceived
| increase in engagement.
| chundicus wrote:
| What are some examples good KPIs and A/B tests for better
| consumer user experiences? Engagement is obviously deeply
| flawed if a good consumer user experience is your goal, but it
| does have the nice property of being easily measured. Do you
| rely on users constantly rating their experience on a numeric
| scale?
| maltelandwehr wrote:
| ,,Messing up your competitors A/B test" is not unheard of as a
| tactic in highly competitive ecommerce settings.
| withinboredom wrote:
| Do software engineers actually implement that? That seems
| pretty immoral. I'd rather let them run the a/b test and
| steal whatever solution they end up with.
| sudosysgen wrote:
| In some contexts you do that or you're fired. Some people
| can't afford to be fired, so they do it.
| cortesoft wrote:
| Eh, I am sure you could convince yourself it isn't
| immoral... everyone in HN seems to think things like google
| analytics are bad because of the privacy implications, and
| doesn't have a problem blocking them (which would also
| 'mess with a/b tests'). You could just argue that you are
| hindering their user spying.
|
| Not a great argument, but good enough to allow a developer
| to sleep at night.
| amelius wrote:
| It would be cool if Google could manage to become a storefront
| for the entire web, thereby eliminating Amazon.
| murgindrag wrote:
| For Google (or anyone) to become a storefront for the entire
| web, they'd need to handle scams (and errors) well.
|
| eBay is a cesspool. Aliexpress is worse. Random web sites are
| bad. Amazon isn't perfect, but it's better.
|
| Amazon also has customer service; they've always made me whole.
| Random web sites, I'm basically SOL. Aliexpress and eBay are
| random. Someone flips a coin, heads seller wins, tails buyer
| wins, regardless of who the scammer is.
|
| I mostly buy from Amazon since my odds of not having problems
| are that much higher.
| ihumanable wrote:
| Exactly this, the customer service for the average consumer
| from Amazon is very difficult to beat and is Google's biggest
| weakness.
|
| Bought some cables from Amazon Basic, one ended up not
| working, another had some cosmetic damage but works fine.
| They refunded both, sent out replacements, and just told me
| to discard them, it wasn't worth it for Amazon to pay to have
| it shipped back.
|
| Of course if you abuse this too much Amazon will ban you. If
| you are an honest consumer though, their customer service
| generally provides a great experience.
|
| I still remember a time when everyone was afraid of
| purchasing stuff over the internet, Amazon has so greatly
| reduced the friction and concern that sometimes I find myself
| going from "hmm, I need something" to "it will be here
| tomorrow" in the matter of a minute or two.
|
| Although more competition in this space would ultimately
| benefit the consumer, it seems unlikely that Google is going
| to be the source of that competition. They've got shopping
| results integrated into their search engine, and it's a
| feature I've maybe browsed from time to time, but I often
| just end up searching and purchasing on amazon directly. I
| don't know if I would be super comfortable purchasing from
| Google in the same way that I am with Amazon, too many horror
| stories of App Developers / YouTube Creators / etc getting
| caught in some sort of Machine Learning Customer Support
| system.
|
| Curious if others use the Google Shopping thing in the search
| engine and what their experiences are with it.
| ksk wrote:
| Between the two, personally, I would rather an advertising
| company like Google be eliminated, or at-least be regulated to
| protect user privacy. For me, it's easy enough to avoid Amazon,
| harder to avoid Google since every website uses them to spy on
| users.
| belval wrote:
| You and I have a very different definition of "cool".
| advisedwang wrote:
| http://archive.is/YRkQe
| maltelandwehr wrote:
| Thanks! I was not aware you could use Web Archive for that. All
| the more reason to Love that site!
| [deleted]
| kqr wrote:
| I'm not sure archive.is and archive.org are the same site.
| mobilio wrote:
| They're not same!
| Keyframe wrote:
| I think I've seen most Google's technologies dissected and/or
| explained in detail over the years. Lots of their own papers too.
| If you look into how and what they're doing regarding data
| collection, including scraping, there's nothing.
___________________________________________________________________
(page generated 2020-07-01 23:00 UTC)