[HN Gopher] A one-year moratorium on police use of Rekognition
___________________________________________________________________
A one-year moratorium on police use of Rekognition
Author : robbiet480
Score : 166 points
Date : 2020-06-10 21:33 UTC (1 hours ago)
(HTM) web link (blog.aboutamazon.com)
(TXT) w3m dump (blog.aboutamazon.com)
| golemiprague wrote:
| So Amazon can do face recognition but police can't? Bezos doesn't
| need police work, he can hide behind his walled house with a
| private security which he will no doubt will sell his recognition
| software to if required, but the rest of you plebs, we will throw
| you to the hands of criminals and predators because of some
| anecdotal incident with a black guy.
|
| Why not stop selling equipment also to hospitals? I have heard
| they also make mistakes, even sometimes negligent doctors killing
| people, only Amazon are pure and don't do any evil.
| m0zg wrote:
| Anyone with a modicum of skill and a few GPUs can do what
| Rekognition does using code freely available on GitHub and public
| datasets. This cat is _way_ out of the bag.
| 1-6 wrote:
| So... cloud service providers now have the right to determine
| what services they want to allow and what they want to shut off?
| HAL 9000: "I'm sorry Dave, I'm afraid I can't do that"
| fwip wrote:
| What do you mean "now?" They've always had the power to do
| that.
| 1-6 wrote:
| Time for some cloud neutrality.
| ricardobeat wrote:
| No, they have always had that right, as does any other private
| business. You can always refuse service if it is not on
| discriminatory grounds (gender, sexuality, ethnicity, religion
| or disability).
| heavyset_go wrote:
| Sexuality is only a protected class in some states, and
| sexuality isn't a protected class at the federal level.
|
| Unfortunately, there are plenty of places in the US where one
| can experience legal discrimination based on their sexuality
| in housing, employment, education, health insurance and
| healthcare. There are several maps here[1] that show where
| those places are. Many of them don't classify violence
| motivated by the victims' sexuality as hate crimes, either.
| There's also a table here[1] that summarizes the
| discrepancies in the US.
|
| [1] https://en.wikipedia.org/wiki/LGBT_rights_in_the_United_S
| tat...
|
| [2] https://en.wikipedia.org/wiki/LGBT_rights_in_the_United_S
| tat...
| brundolf wrote:
| "We are waiting until the current news cycle has blown over"
| [deleted]
| 1-6 wrote:
| This reminds me of an FBI agent who likes to keep his gun in his
| back pocket and have it bulging out, just to impress people who
| knew what he did. I can't imagine how a few bad apples within the
| police force would sit behind a computer playing with Facebook
| profile pictures and matching them against Rekognition.
| boomboomsubban wrote:
| Isn't it open to anyone with an AWS account? So how are they even
| trying to implement this and what would be stopping any third
| party from using this to submit reports to law enforcement.
| [deleted]
| robbiet480 wrote:
| Better than nothing I guess but still not a perfect solution.
| kingo55 wrote:
| It's always a slippery slope when companies morally compel
| themselves to block use cases from their service.
|
| I welcome this move from Amazon, but I hope it doesn't foreshadow
| more moral bans in future e.g. spurred on by the next angry mobs
| who will try to limit free speech in society.
| noway421 wrote:
| Amazing news!
| jonplackett wrote:
| Can someone explain what has kicked off this retraction from face
| recognition from IBM and now Amazon? I mean it's always had
| dubious uses. What has made this happen right now?
| 1-6 wrote:
| It's perfectly acceptable to cash in on using face recognition
| for profit but dubious when used to lock people behind bars
| _sarcasm_
| MattGaiser wrote:
| Opinion of police departments is falling so this could quite
| plausibly be the next big battleground.
| pvaldes wrote:
| Probably that everybody is using masks so the system lacks of
| essential parts and does not work anymore until is rebuilt and
| fixed. There could be also a risk of database being tainted by
| masks with something print over it, like lips, wrinkles in the
| mask, or even the face of somebody.
|
| And of course you can not require citizens to show the face and
| to cover the face at the same time, so to try to publicly
| denounce people with the face covered as delinquent wannabe is
| not possible at this moment.
|
| But I'm just speculating.
| badRNG wrote:
| What is the difference between Rekognition and Clearview AI? I'm
| assuming that Rekognition is just using government photo
| databases rather than social media?
|
| It seems that Amazon has a far better reputation on HN compared
| to Clearview AI. Is that deserved?
| Grimm1 wrote:
| Rekognition is just their normal CV offering you can use for
| anything from what I know.
| MattGaiser wrote:
| Then won't departments just go to the highly secretive companies
| like Palantir?
| superzamp wrote:
| I remember at the AWS summit maybe two years ago, they were
| casually showcasing how some police depts were using Rekognition.
| Oh my, what a culture shock. How can you basically foreshadow
| 1984 on stage without blinking an eye?
| allears wrote:
| Translation: We already made a bundle on this, but we're seeing
| too much pushback, so we're getting out before the downside eats
| into our profits.
| jjeaff wrote:
| Or getting out while black lives matter and will get back in
| when everyone has forgotten again.
| solidasparagus wrote:
| It's reactions like this that make companies not want to even
| try. "People are going to bitch whether we sell to the police
| or not so there is no upside to stopping". Why not have a
| little positivity that AWS is finally restricting access to
| their technology beyond what is legally required?
| zests wrote:
| We should be positive about public pressure having an effect
| on large corporations. We should not be positive about a
| large corporation making the calculated decision that a tiny
| drop in revenue is worth the advertisement and good press
| that comes with it.
|
| What percent of AWS revenue is from Rekognition? Probably a
| rounding error.
| toomuchtodo wrote:
| Go all in then. Disallow the use of the technology without
| independent oversight. But don't do a half hearted measure
| because you're unwilling to commit, or you're going to grasp
| for some positive PR.
|
| > Why not have a little positivity that AWS is finally
| restricting access to their technology beyond what is legally
| required?
|
| Because it's not enough. "Please sir, may I have some more"
| is not how you address the weaponization of technology by a
| trillion dollar org.
| solidasparagus wrote:
| Why do we need to demand all or nothing? AWS is never going
| to allow for independent oversight - not even Google does
| that. Demanding this just means that AWS would never make
| any improvements.
| [deleted]
| sgift wrote:
| Independent oversight? So .. like from a government? Cause
| that's the definition of independent, from the people for
| the people. And if the next answer is "that's not how
| government is in reality" neither is any other
| 'independent' oversight. Humans are easily corruptible. We
| better find a solution for this problem or all that will
| happen is an endless line of watchers watching other
| watchers.
| toomuchtodo wrote:
| _Humans are the only solution_. Rational humans are never
| going to rely on technology alone for enforcement,
| governance, and /or oversight. If you have a problem with
| the humans currently making decisions, find better
| humans. Checks and balances.
|
| Don't like the Big Tech corporate surveillance state?
| Write better laws regulating them. Don't like the people
| writing laws currently? Vote and run against them. Still
| not heard? There are yet more avenues for recourse.
|
| The idea that technology is going to fix these problems
| holds no basis in reality.
| sgift wrote:
| Seems I misunderstood your previous post, cause
| everything you wrote I agree with completely.
|
| My only point here is: Currently, people do not trust
| independent oversight (read: government). And there are
| probably a few good reasons. So, I don't see how saying
| "Amazon should only sell this technology with independent
| oversight" fixes anything as long as the trust problem
| isn't solved.
| toomuchtodo wrote:
| Definitely talking past each other. You must solve for X,
| where X=trust.
| lewiscollard wrote:
| > Independent oversight? So .. like from a government?
| Cause that's the definition of independent, from the
| people for the people.
|
| If you do truly hold the belief that governments are "the
| definition of independent", and that all governments act
| solely in the best interests of their own
| citizens...well, to borrow some words from Public Enemy:
| Can't do nuttin' for ya man.
|
| We're literally talking about the government using facial
| recognition with no oversight and little public debate
| and no consent from the public, so you wouldn't have an
| excuse for coming out with this "government is all of us
| working together" lorem ipsum even if you were born 30
| seconds before the article was posted and didn't know
| anything about the way that governments actually behave
| in practice.
|
| You come so very close to getting it, in that you
| acknowledge that this isn't how governments work in
| reality, and then you're like "well neither is anything
| else though" as if that is somehow an argument for a
| strategy that you kinda-acknowledge cannot work.
|
| And FWIW: I agree, it's fucked, it needs to stop, both in
| the public and private sector, and anyone that works on
| such tech should be shunned. But your assumptions are
| false to fact.
| [deleted]
| Barrin92 wrote:
| because there's generally much cynicism about piecemeal
| measures in regard to surveillance. Over the past, it's been
| exceedingly common that both private firms, as well as
| legislators, halt some program or legislation only to bring
| it back a year or two later.
|
| Arguably we need a much more principled, stronger stance on
| opposition to mass surveillance period. Companies that
| understand the ethical obligations should get out of it
| completely.
| claudeganon wrote:
| A couple months ago, Amazon execs were reported to be
| conspiring to smear a black labor organizer with racist dog
| whistles.
|
| https://www.vice.com/en_us/article/5dm8bx/leaked-amazon-
| memo...
|
| Treating their motives with anything but the utmost cynicism
| seems the rational move here.
| solidasparagus wrote:
| Bottomless cynicism gets us nowhere. It's far from done,
| but unless there's some evidence that they are still
| selling Rekognition to the police, this is a positive
| development in the field of facial recognition.
| noway421 wrote:
| > We already made a bundle on this
|
| Closing a profit generating business line is all more difficult
| than closing an unproven one. Props to folks at Amazon, this a
| change for the better.
|
| I'm wary about their competitors though. Looks like an
| opportunity for Microsoft to monopolise the face recognition
| software market. Can't think of any market force to solve this,
| and regulation would probably leave a lot of room to game the
| system.
| slg wrote:
| It is a one year moratorium. They aren't even getting out for
| good. They are pausing it, hoping this issue will be forgotten
| about it a year, and then they will resume making money off it.
| lgleason wrote:
| Pretty soon you will see telephone companies refusing service,
| car companies refusing to service cars etc. based on a moral
| judgement of the customer. Then whey will finish the process of
| getting anybody accused of wrong-think purged from employment
| etc.. Hmmmm, where is history was that tried before? What could
| possibly go wrong?
| amaccuish wrote:
| You mean like baking cakes? Conservatives can't have it both
| ways.
| boustrophedon wrote:
| This is already the case today under capitalism. A business may
| refuse service to anyone provided they are not a member of a
| protected class (the Civil Rights act and the ADA define a
| few).
| DaniloDias wrote:
| It's more disturbingly the case under whatever economy China
| runs. https://foreignpolicy.com/2018/04/03/life-inside-
| chinas-soci...
| [deleted]
| 1-6 wrote:
| Gofundme.com
| woodruffw wrote:
| As we all know, every moral issue has a 1-year expiration date.
| cle wrote:
| Every moral issue is also black-and-white and requires no
| practical (or ethical) implementation plan.
| 1-6 wrote:
| Ahh, one more year until I finish creating my fake digital self
| and delete fingerprints.
| Terretta wrote:
| It's probably the legal and PR teams' fault, but this surely
| could have been worded to sound less like potential corporate
| doublespeak:
|
| _" We've advocated that governments should put in place stronger
| regulations to govern the ethical use of facial recognition
| technology, and in recent days, Congress appears ready to take on
| this challenge. We hope this one-year moratorium might give
| Congress enough time to implement appropriate rules, and we stand
| ready to help if requested."_
|
| Dead giveaway is that Legal and PR teams relentlessly edit out
| self-agency.
| th0ma5 wrote:
| What opportunities exist for use of facial recognition by protest
| groups and citizen watchdogs? Any products? Any success stories?
| deegles wrote:
| "We've advocated that governments should put in place stronger
| regulations to govern the ethical use of facial recognition
| technology"
|
| Or you know... you could do it yourself. Ethics don't have to
| come from regulations.
| everfree wrote:
| They're hamstrung by their shareholders. This is why it's
| impossible to find a public company that cares about ethics in
| any context other than potential blowback.
|
| Amazon can keep the sharks at bay for a year and cry for help,
| but if regulation is too late, they're going to be eaten for
| lunch by shareholders and they know this.
| throwawaysea wrote:
| Personally I am pro facial recognition being used, and believe it
| is very necessary if police budgets are cut and patrols are
| reduced. We need a way to hold criminals accountable and bring
| them to justice. Cameras with facial recognition let us identify
| their location and send police officers to apprehend criminals,
| instead of relying on the random chance that an officer spots
| someone while driving around and matches their face with a list
| of perps they've seen before.
|
| I haven't heard of any police departments using facial
| recognition as a definite match. All of them use human
| confirmation. So basically, the number of false positives does
| not matter - it's more that facial recognition reduces the total
| amount of data to a more manageable number that are scrutinized
| by human eyes.
|
| I don't know why people would be against this. Steps like this
| moratorium just seem like posturing or an overreaction. The
| recent policing incidents that have been in the news do not
| involve facial recognition and there is no reason to tie them in.
| ipsocannibal wrote:
| Step 2: End Ring contracts with police departments.
|
| https://www.eff.org/deeplinks/2019/08/five-concerns-about-am...
| chishaku wrote:
| The ACLU is doing a lot of great work to hold government
| accountable when it comes to facial recognition tech.
|
| https://www.aclu.org/press-releases/aclu-challenges-fbi-face...
|
| https://www.aclu.org/press-releases/aclu-challenges-dhs-face...
|
| Would be great to see Amazon's support.
|
| The ACLU ran an experiment with Rekognition and these are their
| findings:
|
| "Using Rekognition, we built a face database and search tool
| using 25,000 publicly available arrest photos. Then we searched
| that database against public photos of every current member of
| the House and Senate. We used the default match settings that
| Amazon sets for Rekognition.
|
| ... the software incorrectly matched 28 members of Congress,
| identifying them as other people who have been arrested for a
| crime.
|
| ... Academic research [0] has also already shown that face
| recognition is less accurate for darker-skinned faces and women.
| Our results validate this concern: Nearly 40 percent of
| Rekognition's false matches in our test were of people of color,
| even though they make up only 20 percent of Congress."
|
| https://www.aclu.org/blog/privacy-technology/surveillance-te...
|
| [0]:
| http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a...
| hnmullany wrote:
| I'm a big ACLU supporter, but thought this was poorly done.
| They never released either the database or the code for this
| testing, and had configured the recognition level against
| Amazon's recommendations.
| chishaku wrote:
| > had configured the recognition level against Amazon's
| recommendations.
|
| Citations?
|
| My understanding was that the ACLU used the default settings.
|
| July 26, 2018 -- Amazon states that it guides law enforcement
| customers to set a threshold of 95% for face recognition.
| Amazon also notes that, if its face recognition product is
| used with the default settings, it won't "identify[]
| individuals with a reasonable level of certainty."
|
| July 27, 2018 -- Amazon writes that even 95% is an
| unacceptably low threshold, and states that 99% is the
| appropriate threshold for law enforcement.
|
| https://www.aclu.org/press-releases/aclu-comment-new-
| amazon-...
|
| Either way, the defaults are the problem if the application
| is law enforcement.
|
| "Defaults have such powerful and pervasive effects on
| consumer behavior that they could be considered "hidden
| persuaders" in some settings. Ignoring defaults is not a
| sound option for marketers or consumer policy makers. The
| authors identify three theoretical causes of default effects
| --implied endorsement, cognitive biases, and effort..."
|
| https://journals.sagepub.com/doi/10.1509/jppm.10.114
| gregsadetsky wrote:
| In ACLU's posts, they say that they used the "default match
| settings". In their response [0] to Amazon's response they
| link to a guide published by Amazon [1] intended to "Identify
| Persons of Interest for Law Enforcement" that does use
| `searchFaceRequest?.faceMatchThreshold = 0.85;`
|
| Fast Company [2] writes about this as well: "The ACLU in both
| tests used an 80% match confidence threshold, which is
| Amazon's default setting, but Amazon says it encourages law
| enforcement to use a 99% threshold for spotting a match."
| That bit of the article links to the CompareFaces
| documentation [3] which states "By default, only faces with a
| similarity score of greater than or equal to 80% are returned
| in the response".
|
| Have you seen/read something else about this?
|
| [0] https://www.aclu.org/press-releases/aclu-comment-new-
| amazon-...
|
| [1] https://aws.amazon.com/blogs/machine-learning/using-
| amazon-r...
|
| [2] https://www.fastcompany.com/90389905/aclu-amazon-face-
| recogn...
|
| [3] https://docs.aws.amazon.com/rekognition/latest/dg/API_Com
| par...
| tw000001 wrote:
| >Fast Company [2] writes about this as well: "The ACLU in
| both tests used an 80% match confidence threshold, which is
| Amazon's default setting, but Amazon says it encourages law
| enforcement to use a 99% threshold for spotting a match
|
| Then this whole thing is potentially misleading because
| there's a huge difference between 80% and 99%. It's
| probably nonlinear and they could possibly see their false
| matches drop to 0. This is not a fair test - or rather, the
| conclusions are not quite supported by the parameters.
|
| Not that I'm defending police use of facial recognition
| tech, I think it's abhorrent, though possibly inevitable.
| JMTQp8lwXL wrote:
| It seems realistic other users would ignore Amazon's
| recommendations for proper configuration.
| schwarzmx wrote:
| > Would be great to see Amazon's support.
|
| From one of the links on the left of the article:
| https://blog.aboutamazon.com/policy/amazon-donates-10-millio...
|
| > Update, June 9: Since announcing our $10 million donation,
| we've heard many employees are making their own contributions--
| and we've decided to match their donations 100% up to $10,000
| per employee to these 12 organizations until July 6, 2020.
| chishaku wrote:
| If so inclined, support the ACLU:
|
| https://action.aclu.org/give/donate-to-aclu
| ve55 wrote:
| The ACLU used to be on the top of my list of my personal
| favorite charities, but I've found they've become
| significantly more reactionary in the last few years,
| prioritizing emotionally outraging causes over utilitarian
| causes, so don't donate to them any longer.
|
| With that said, some of their work is still great, and I'm
| thankful for it.
| xrd wrote:
| I'm scared of that outcome. I'm glad you donated to them in
| the past so you obviously valued them at some point. Do you
| have articles to back that up? I'm not trolling; I wonder
| if this is the message that the "powers-that-be" want us to
| think.
|
| Remember how McDonalds was the victim of a baseless
| lawsuit? Well, it wasn't actually the case, but that sure
| benefitted corporations who can now assert most lawsuits
| against them are frivolous.
|
| https://www.caoc.org/?pg=facts
| roenxi wrote:
| From your link:
|
| > McDonald's operations manual required the franchisee to
| hold its coffee at 180 to 190 degrees Fahrenheit.
|
| > Coffee at that temperature, if spilled, causes third-
| degree burns in three to seven seconds.
|
| They might also be interested to know it was composed
| primarily of dihydrogen monoxide, a lethal chemical agent
| known to be the proximate cause of hundreds of deaths
| each year. They are describing boiling water.
|
| Even if McDonald's did the wrong thing, it is a frivolous
| lawsuit. I serve boiling hot tea to all my guests, it
| isn't negligence. The case as described seems to be that
| McDonald's should be civilly liable for serving a hot
| beverage in a styrofoam cup to a customer who asked for a
| hot beverage and could easily detect it was in a
| styrofoam cup.
| Karunamon wrote:
| Having read extensively on that case, my opinion changed.
| I now consider the lawsuit baseless and the verdict
| primarily the result of a sympathetic plaintiff and an
| unsympathetic defendant (the stereotypical sweet old lady
| vs the evil money-grubbing megacorp).
|
| Their coffee is just as hot nowadays and to lower the
| temperature to the degree to where the effect on on
| Stella would have been meaningfully different would
| result in lukewarm, under-extracted coffee that fewer
| people would be interested in. Further, the sheer
| quantity of Mcdonalds coffee moved every year without
| incident implies user error, rather than product error.
|
| Yes, I know what the jury said and how they divided up
| the blame. I disagree with their concluison.
| winstonewert wrote:
| Its true that the McDonald's case is often presented in a
| one-sided fashion that makes it look like it was a
| baseless lawsuit.
|
| But its also true that responses, like the one you linked
| to are also often one-sided. And what would you accept
| from an association of lawyers who make money launching
| such suits?
|
| The truth is, its not as simple as either side tries to
| make it out to be. I think the Wikipedia article: https:/
| /en.wikipedia.org/wiki/Liebeck_v._McDonald%27s_Restau...,
| does a good job of presenting pertinent details from both
| sides.
| casefields wrote:
| >Ira Glasser says the organisation he once led has
| retreated from the fight for free speech.
|
| The ACLU would not take the Skokie case today':
| https://www.spiked-online.com/2020/02/14/the-aclu-would-
| not-...
|
| Former ACLU board member Wendy Kaminer:
|
| The ACLU Retreats From Free Expression:
| https://www.wsj.com/articles/the-aclu-retreats-from-free-
| exp...
|
| to complaints of sexual violence. We will continue to
| support survivors."
|
| The ACLU Declines to Defend Civil Rights:
| https://www.theatlantic.com/ideas/archive/2018/11/aclu-
| devos...
| throwawaysea wrote:
| I supported them for a long time as well, but they've waded
| very far from their original waters. These days the ACLU is
| incredibly politically biased, and the issues they focus on
| are often non-issues or low-priority ones. I am
| disappointed to see them, for example, file lawsuits
| against schools that are trying to ensure that only
| biological women participate in sports divisions for women,
| so that those sports are competitive and fair.
|
| The ACLU's social media accounts are a mess as well. Their
| postings come off as unhinged and sue-happy, and the fan
| base of commenters has become so one-sided, that I think
| the ACLU simply caters to that vocal audience now. Maybe
| the change is not solely attributable to that - there might
| also be a new generational wave of inside actors that
| simply operate the ACLU in a more ideological manner.
|
| I agree that some of their work is still great. But
| unfortunately it's been enough of a change that upon
| weighing the good and bad, I had to finally pull the plug
| on my recurring donations too.
| tw000001 wrote:
| >These days the ACLU is incredibly politically biased
|
| I was going to post the same in anticipation of the
| partisan downvotes that you're receiving. Without going
| into specifics, this sums it up nicely:
|
| >It's not that the left shouldn't have opportunities to
| speak up against the president's agenda -- of course it
| should. But the ACLU shouldn't be its political bullhorn.
| The organization's legal independence gave it special
| standing. By falling in line with dozens of other left-
| leaning advocacy groups, the ACLU risks diminishing its
| focus on civil liberties litigation and abandoning its
| reputation for being above partisanship
|
| One issue in particular is the ACLU's interpretation of
| the second amendment, which they do not fight for with
| the same fervor as the first.
|
| 1.https://www.realclearpolitics.com/articles/2018/02/08/t
| he_ac...
| ikeyany wrote:
| Why do they say Congress is ready to take on this challenge?
| Congress hasn't passed a thing yet.
| thephyber wrote:
| "ready to take on" sounds more like the beginning of the
| funnel.
|
| "hasn't passed a thing" sounds like the end of a Congressional
| funnel.
|
| edit: this just purely a retort about the specific complaint of
| the parent. I don't deny that Congress hasn't actually done
| much useful to forward the policy changes I would deem
| desirable here.
| [deleted]
| siruncledrew wrote:
| What prevents a private company (ex. Clearview) from using
| Rekognition to accomplish the same thing for the police as a
| government-contractor?
|
| Without any kinds of laws, wouldn't things like this incentivize
| new niches to popup to milk money from the government?
| ErikAugust wrote:
| Has anyone ever here actually demoed Rekognition? I did two years
| ago, maybe.
|
| From that, I felt like it doesn't work and shouldn't be used in
| production, never mind police production.
| coderintherye wrote:
| Yes, we ran it against our borrower image set and the results
| were atrocious, those this was now a couple years past. Never
| tried again after that.
| ciarannolan wrote:
| > we ran it against our borrower image set
|
| Why?
| [deleted]
| rhema wrote:
| You don't really need facial recognition once you have enough
| data from contact tracing.
| dfsegoat wrote:
| AWS is running a nice screen here. I recall reading about
| Rekognition being documented as having accuracy problems when
| individuals in question had darker skin [2,3].
|
| >> _" The latest cause for concern is a study published this week
| by the MIT Media Lab, which found that Rekognition performed
| worse when identifying an individual's gender if they were female
| or darker-skinned."_ [1]
|
| I can't really comment. Just recalled this in the memory banks
| and thought they might address this directly [they may have].
|
| 1 - https://www.theverge.com/2019/1/25/18197137/amazon-
| rekogniti...
|
| 2- https://www.media.mit.edu/articles/amazon-is-pushing-
| facial-...
|
| 3- https://www.marketwatch.com/story/ai-experts-take-on-
| amazon-...
| hpoe wrote:
| There was a discussion on HN last week that I can't find where
| I was enlightened to find out that one of the big problems
| around doing facial recognition is proper lighting and without
| it you can't really build good models and be able to really use
| the image.
|
| As an extension of this photographs of individuals with darker
| skin required more lighting than photographs of individuals
| with darker skin.
|
| I don't know all of what goes into the ML for facial
| recognition and I am sure there are people far smarter than me
| working on it (and making way more money than me to boot), but
| I guess my thought here is that some variation of Poe's Law
| applies. I know that people are quick to jump to condemn
| something as racist but sometimes there really are just honest
| mistakes.
|
| I have a hard time believing that anyone at any level of the
| AWS structure set out to produce a racist facial recognition,
| but rather it may have just been an honest over site and rather
| than rushing to crucify them we should instead look at it as a
| learning opportunity to help develop further the field of
| facial recognition.
| akiselev wrote:
| The technological implementation is almost certainly an
| honest mistake but the marketing and sales around it is not.
| We've known about the importance of lighting in photography
| for more than a century and the computer vision industry has
| been battling the issue for decades in manufacturing because
| metal is shiny. Kodak even used to manufacture different
| films to better represent non-white people and camera sensor
| manufacturers today have entire teams with a swarm of diverse
| models dedicated to accurate color reproduction of skin tones
| across races. It's been an ongoing issue for people of color
| ever since TV became a thing because those technologies were
| initially very poor at color reproduction as well, but that
| didn't stop newscasters from jumping face first into
| televised manhunts.
|
| We've been around this block several times before and while
| the quarry may change from photo accuracy to ML driven facial
| recognition, the hunt does not. There's no excuse for selling
| technology to industries facing (or creating) life and death
| situations when the bugs are so obvious.
| coderintherye wrote:
| This is EXACTLY what people are trying to raise awareness of.
| This is implicit bias. It doesn't matter if people have "good
| intentions" or made "honest mistakes" if the tool is
| implicitly biased. This is why having a diverse team working
| on a project is important, because it will help call out
| these issues early rather than after they have been put in
| production.
|
| It's also not purely a technical problem, if you feed the
| model only pictures of white and asian male college students
| then it's no surprise when you get a model that biases
| towards recognizing white and asian male college students
| (which is exactly how several prominent models were trained).
| zerocrates wrote:
| Biased algorithms/models are particularly dangerous because
| they tend to provide a veneer of objectivity (plausible
| deniability if you want to put a more cynical lens on it)
| that could frustrate attempts to hold users accountable.
| cpeterso wrote:
| Mathematician Cathy O'Neil's book "Weapons of Math
| Destruction: How Big Data Increases Inequality and
| Threatens Democracy" is a good introduction to the implicit
| biases in machine learning:
|
| https://en.wikipedia.org/wiki/Weapons_of_Math_Destruction
| mc32 wrote:
| I think the main thing is that even if the technology were
| improved and it could be proven that the biases were so low
| as to be negligible or that it basically identifies people
| equally well, the opposition would be on the technology as a
| tool.
|
| People want _some_ privacy in public. A tech that can track
| or backtrack people 's movements is kinda creepy in a few
| ways.
| hpoe wrote:
| I see this as an extension of the Facebook <moderating/censoring>
| discussion, which is really a broader question of what moral
| obligations do corporations have beyond following the law and
| trying to provide the optimal product to their consumers?
|
| Also there seemed to be no substantive discussion prior this
| about the police using Rekognition until it became a hot button
| issue. What will the widespread effects be if corporations start
| allowing their decisions to be governed by <outrage of the
| mob/principled consumer pressure>?
|
| Finally I wonder how they will implement this, I mean after all I
| can sign up and start using any AWS service with just a credit
| card what's to stop police departments from simply using a
| corporate card and signing up for a different account? Also does
| this apply to just local PDs or does it extend to the FBI, NSA,
| CIA, or other 3 letter government agencies?
|
| Disclaimer: The purpose of these comments are intended to be
| observational not advocational.
| moron4hire wrote:
| Well, considering that corporations are people now, I would say
| they have all the same moral obligation as people.
| albntomat0 wrote:
| Similarly, I would argue that we don't want to be reliant on
| corporations to determine what the lines are themselves, as
| they should not be de-facto moral authorities. Determining what
| is acceptable needs to come from our society as a whole,
| through reasoned debate and proper functioning of government.
| tdfx wrote:
| > through reasoned debate and proper functioning of
| government.
|
| Yikes. Is there a fallback option?
| albntomat0 wrote:
| The other options seem to be the benevolence of Bezos or
| Twitter condemnation, so not really.
| kmonsen wrote:
| Still better in some cases than relying in government
| taking action :-(
|
| Note that this is a one way relationship, corporations
| must comply with laws, but can also do other things.
| albntomat0 wrote:
| I'd say "better" as in "more likely", but I'm still not a
| fan of a handful of SV folks being the moral decision
| makers for the world as the general order of things.
|
| I'll gladly accept additional benevolence from them
| though! Just not as the sole power in the area.
| banads wrote:
| >what moral obligations do corporations have beyond following
| the law and trying to provide the optimal product to their
| consumers?
|
| Corporations have no such morals. They are profit seeking
| social constructs. Breaking the law is often a profitable cost
| of doing business, as is making an ever increasingly shitty
| product when there is little to no competition.
| darkerside wrote:
| Moral obligations, almost by definition, do not exist. That
| goes for anyone, not just corporations. They are the things
| you should do even though there is no reason to do them.
|
| Arguing corporations have no morals is being pedantic. The
| question is clearly, what moral obligations _should_ they
| have?
| boomboomsubban wrote:
| >what moral obligations do corporations have beyond following
| the law and trying to provide the optimal product to their
| consumers?
|
| This question just does not work when you consider how much
| companies spend lobbying.
| chishaku wrote:
| > there seemed to be no substantive discussion
|
| Just because you don't know about it doesn't mean it doesn't
| exist.
|
| Try google "aclu facial recognition", "eff facial recognition".
|
| congress.gov returns 132 bills introduced in the last two
| sessions going back to 2017. If you read the titles, it's clear
| many of them are related to transparency and respecting rights
| to privacy.
|
| https://www.congress.gov/search?searchResultViewType=expande...
| hpoe wrote:
| I don't mean to imply that there wasn't discussion or
| advocacy around the issue but rather it was not an issue that
| was high in public awareness or concern.
| ncallaway wrote:
| I don't really see how this interpretation of what you
| wrote makes sense.
|
| "Also there seemed to be no substantive discussion prior
| this about the police using Rekognition until it became a
| hot button issue."
|
| If you didn't mean that it wasn't an issue that was high in
| public awareness or concern wouldn't it be tautological
| that it wasn't high in public awareness or concern until it
| was a hot button issue? Like, the definition of it becoming
| a hot button issue is that it's high in public awareness or
| concern.
|
| Am I misinterpreting something? Did you mean something else
| and I just got it wrong?
| monsonjeremy wrote:
| I think it's an absurd assertion to say "there was no
| substantive discussion prior to this about police using
| Rekognition". Seems to me that people are now becoming aware of
| the problem with this, and Amazon has implemented a one-year
| (not permanent moratorium) in order to allow for more
| discussion and legislation around the issue. There are much
| bigger implications than the facebook issue if you are
| misidentifying criminals as a result of bad facial recognition,
| so you can't really equivocate the two.
|
| Also, to call this "mob outrage" or "principled consumer
| pressure" is delegitimizing the entire thing. Do you really
| genuinely think this happens any other way? Seems like when a
| lot of people start to have a problem with something, it makes
| sense to have a moratorium and investigate
| improvements/solutions.
| kmonsen wrote:
| I don't think there is any obligation to do more than follow
| the law. I believe Amazon is here trying to go one step above
| and do what it believes is right. Clearly corporations are
| allowed to put more stringent requirements on itself if it
| believes that is right.
___________________________________________________________________
(page generated 2020-06-10 23:00 UTC)