[HN Gopher] Live-caption glasses let deaf people read conversati...
___________________________________________________________________
Live-caption glasses let deaf people read conversations using
augmented reality
Author : vinnyglennon
Score : 164 points
Date : 2023-03-14 13:41 UTC (9 hours ago)
(HTM) web link (www.youtube.com)
(TXT) w3m dump (www.youtube.com)
| autoexec wrote:
| What are the privacy/security issues with this? Does this mean
| every conversation a person wearing these has (or that occurs
| within earshot) is being collected and harvested by someone? Will
| the AR be used to insert ads into people's conversations or
| plaster images of ads all over the place? Will certain words or
| phrases be automatically censored?
|
| This is cool tech, that could be used to help people, but it
| comes with lots of potential for new forms of evil that were not
| possible without it. Considering that I can't remember the last
| time I bought a product using a new technology that wasn't also
| designed to work against my interests, I'm immediately skeptical
| of any device that can't be used offline and especially one that
| requires being connected to cell phone apps.
| vlunkr wrote:
| > Will the AR be used to insert ads into people's conversations
| or plaster images of ads all over the place?
|
| I doubt it. This isn't the kind of cheap mass-market device
| where running ads is going to make you a big profit.
| lazyeye wrote:
| Completely agree. There seems to be such a gap in the market
| for privacy-first hardware but very few companies are doing
| anything.
| Kalium wrote:
| The key question is always a painfully simple one: how much
| are people willing to pay as a privacy-first premium?
|
| The answers to this determine everything. Treating privacy-
| first as the moral and ethical default we should expect
| everyone to start from is a wonderful idea, rooted in
| compassion, kindness, and a foundational respect for human
| rights. It has also been an abject failure to date.
|
| We should not expect the future to be different unless we are
| willing to be realistic about the economics at work.
| Otherwise the market gap will remain in the realm of the
| wonderfully hypothetical forever.
| OmahaBoy69 wrote:
| Shameless plug:
|
| https://github.com/TeamOpenSmartGlasses/OpenSourceSmartGlass.
| ..
|
| ASR is done locally on the user's phone.
| Rustwerks wrote:
| The implications are approximately the same as
| https://www.amazon.com/tape-recorder/s?k=tape+recorder except
| with compute power to make storage and search more convenient.
| Except that the glasses/app don't save anything by default nor
| do they need to use the internet.
|
| I suppose it would be a nice feature if they saved all of your
| conversations for later? The translations are too imperfect for
| any legal matter use.
| Vrondi wrote:
| Bad choice of example, because a tape recorder has no network
| connection and requires some effort to capture into a digital
| format that can easily be shared on a network. Unlike tape
| recorders, recent digital devices and apps often harvest your
| info behind the scenes.
| Sunspark wrote:
| It's not that easy to pick up every speech utterance in a wide
| range and separate it by speaker. The further away the sound
| is, the less intelligible it is. An artificial non-directional
| microphone is unlikely to pick up with the same clarity or
| distance as with your own ears. There should not be any privacy
| concern with having a microphone ear vs a biological ear. If
| there is a concern the best way to manage it is to not talk
| about confidential topics with other people in the room.
|
| There won't be any ads, because people would just use their
| phones instead which don't have ads. Specifically Google Live
| Transcribe, Otter and the like. Those require a data connection
| to the network, but there are versions that don't need the
| network at all. E.g. Chrome's Live Caption option. Eventually
| as technology becomes more power efficient and miniaturized it
| won't need to be paired to a phone.
|
| The advantage of glasses is that people find it very
| distracting seeing a phone scrolling away, my GP stares at the
| phone instead of me because he is fascinated by it. Sometimes
| you can't be holding a phone up if something is being worked
| on. The glasses would also allow for a bit more directionality.
| It's a promising tool depending on how well it is implemented.
| primax wrote:
| I don't see how this is inherently worse than mobile phones,
| which are carried by everyone in the developed world all the
| time.
| hunter2_ wrote:
| The likelihood that what you say "in person" is recorded and
| stored by someone easily subpoenaed increases quite a bit.
| While mobile apps might record and upload without the owner
| of the phone knowing, the odds seem low; this changes the
| situation significantly.
| danscarfe wrote:
| We've taken a privacy-first approach here. All data is only
| ever stored on the device, owned by the user, inaccessible to
| us. It's only ever transcribing when a user asks it to and only
| stored if the user asks it to be. We don't censor anything. We
| are soon to release purely on-device transcription, but the
| quality of this is still not as good as the cloud providers
| offer. The app itself is what powers the glasses, they are just
| output devices.
| ghotli wrote:
| So I dug around a little bit and figured I'd just ask.
|
| As just some guy on the internet, can I buy one of these and
| write a hello world to have text show up in front of my eyes of
| my own choosing? Does it have an API or will it?
| zoklet-enjoyer wrote:
| My Pixel phone can do this. Why are the glasses so expensive and
| bulky?
| NovaDudely wrote:
| 1. They don't have the same kind of budget as Google to get the
| scale down as effectively.
|
| 2. If these are the ones I have heard about before, all the
| speech to text is done on the device not in the cloud. This is
| for privacy reasons. Means it needs a bit more bulk for the
| gear to work.
| zoklet-enjoyer wrote:
| These glasses are connected to a phone app
| danscarfe wrote:
| Compared to contemporary AR glasses ($2k+), these Nreal Airs
| are fairly cost effective at $379. We plan to support lots more
| glasses as they come out, especially ones with waveguide
| technology that doesn't require the shared lenses.
| ElijahLynn wrote:
| Video doesn't have a link to product. Here it is:
| https://xrai.glass/
| WastingMyTime89 wrote:
| Reading the comments here, I think people are missing how many
| people are losing hearing while aging and how alienating it is.
| Even if it only works with one speaker at a time, it could mean a
| massive quality of life improvement for a large and growing share
| of the population.
|
| Sure it won't solve the issues faced by the deaf community but
| that's only a tiny portion of the people handicapped by
| difficulty hearing.
| eclipxe wrote:
| https://www.youtube.com/watch?v=SQd394a4qEo
| flanbiscuit wrote:
| Anyone know the status of these Google AR glasses?
|
| The form factor of Google's AR glasses look much much closer to
| a normal pair of glasses than the glasses in the top video
| (which look like heavy sunglasses with a wire connecting to
| your phone)
|
| You posted a CNET link, here's a direct link to Google's:
| https://www.youtube.com/watch?v=lj0bFX9HXeE
|
| In the description it says: "This device has not been
| authorized as required by the rules of the Federal
| Communications Commission. This device is not, and may not be,
| offered for sale or lease, or sold or leased, until
| authorization is obtained."
|
| So I guess they might be waiting for that?
| giraffe_lady wrote:
| Very cool, especially for people who lose their hearing later in
| life. For other deaf people it's important to remember that
| written english is not a form of their native sign language1, so
| this would be like (because it is) reading captions in a second
| language. Still potentially useful but with more limitations. Not
| that there's necessarily a technological way around those
| limitations either.
|
| 1 Afaik this applies to all other sign languages outside english
| too. Signed Exact English exists and probably other-language
| equivalents too but I've never met a native speaker.
| generalizations wrote:
| The problem with ASL compared to lip reading is that it's a
| form of self-segregation, limiting the deaf person to primarily
| communicating only with other people who know ASL. If these
| glasses are effective, it could help bridge that gap.
| [deleted]
| giraffe_lady wrote:
| Why would you compare an entire language to a single
| technique like that? There is no "the problem" with ASL any
| more than there is "the problem" with english or any other
| language. Yes communicating across the barrier can be a
| challenge but that's just the nature of having more than one
| language in use.
| generalizations wrote:
| > that's just the nature of having more than one language
| in use
|
| Well, yeah. I guess my point is that ASL is pretty much a
| foreign language.
|
| I'd compare it to the situation with the english language
| worldwide - since english is the _lingua franca_ , so to
| speak, many countries around the world teach it as a second
| language. If you don't learn english, then (generally
| speaking) you're at a disadvantage because you can only
| communicate with a subset of your population.
|
| I'm not saying there's a problem with deaf people, any more
| than there's a problem with anyone else who simply doesn't
| happen to know the languages of some of the people around
| them.
| TehShrike wrote:
| Only being able to see ASL is different from speaking a
| different language - it's still English. Not being able to
| hear is more like not being able to read.
|
| It's the same language you already know, but you're missing
| out on one of the primary ways people use it.
| metal_am wrote:
| ASL is based on French, so the structure is completely
| different.
| giraffe_lady wrote:
| ASL is not based on french either. It's related to LSF,
| the sign language used in france, but that _also_ isn 't
| based on french. The modern sign languages emerged among
| deaf populations and have completely different grammar
| and morphology from the spoken languages of the cultures
| surrounding their origins.
| metal_am wrote:
| Thank you for correcting me. That's a deep rabbit hole
| giraffe_lady wrote:
| Oh you should definitely look it up then because this is
| completely incorrect. Sign languages are fully distinct
| languages with their own histories and influences.
|
| For example the sign languages spoken in the US and in
| the UK have different ancestries and are _not_ mutually
| comprehensible, despite both countries using english as
| spoken languages.
| elliekelly wrote:
| No, ASL is not English. ASL is a complete and distinct
| natural language in its own right.
| elliekelly wrote:
| I suspect (and hope) you had no ill intent but this comment
| is really ignorant. There is a terrible history of Deaf
| people being discriminated against and forced to "lip read"
| rather than communicate through ASL. And by "forced" I mean
| they were more or less mentally and physically tortured into
| compliance.
|
| Your comment not only perpetuates this totally false
| narrative that there's a "problem" with ASL but it makes it
| sound like Deaf people have chosen only to socialize among
| themselves when the reality is that we have built a world
| that makes communication difficult for Deaf people. It
| doesn't have to be that way:
| https://icyseas.org/2014/01/12/marthas-vineyard-deaf-
| people-...
|
| You might find the Deaf mythology(?) of Eyeth interesting:
| https://www.nytimes.com/2021/10/10/opinion/deaf-
| population-i...
| generalizations wrote:
| I feel like you're making my comment out to be a lot more
| hostile than it was intended.
|
| I don't see any "problem" with deaf people. I see ASL as,
| effectively, a foreign language; and it makes sense to me
| that in general, you're able to live more effectively when
| you can speak the same language as the people around you.
|
| When I traveled overseas to a spanish-speaking country, I
| learned spanish so that I could communicate with the people
| there. It would be unreasonable for me to show up as the
| cliche american tourist and expect the spanish-speaking
| people there to learn english so that I could communicate
| with them.
| PuppyTailWags wrote:
| I think you're misunderstanding the parent poster: You
| don't have to be intentionally hostile to perpetuate
| harmful and ignorant falsehoods. In fact the parent
| poster states they don't believe you're intentionally
| perpetuating ignorance. But, it's reasonable for someone
| to firmly rebut harmful ignorance in a manner, even if
| that level of firmness doesn't match your self-perception
| of your ignorance.
| generalizations wrote:
| Which of us is ignorantly perpetuating a falsehood? It's
| important to maintain civil dialog until the truth is
| discovered.
|
| Edit: got annoyed.
| PuppyTailWags wrote:
| The poster in question said you were ignorant and linked
| to multiple places to give you more context.
|
| edited: the below sections refers to something that has
| been edited out
|
| I understand that it might feel harsh, but to me, it
| looks like you're just being corrected and rebutted just
| like you said you'd be open to.
| generalizations wrote:
| I did read the links. And my response incorporated what
| I'd understood from those. I was actually kinda curious
| how the parent would respond to the analogy with foreign
| travel.
|
| Rebuttals aren't just one-and-done. You have to be able
| to actually defend your position, and that requires a lot
| more than just one statement.
| giraffe_lady wrote:
| > Which of us is ignorantly perpetuating a falsehood?
|
| lmao it is absolutely you that's what we're trying to
| tell you
| generalizations wrote:
| That's the problem with logic; it works both ways. If you
| were the ignorant one perpetuating the falsehood, how
| would you know unless I told you?
| giraffe_lady wrote:
| Wait are you implying you think the main way people learn
| things is by spewing bullshit on the internet until
| someone contests it? Read a book ya dingus. wikipedia.
| christ.
| generalizations wrote:
| That makes no sense. And I have less confidence than
| before that you know what you're talking about.
| elliekelly wrote:
| I'm not sure how you can read hostility into my comment
| when I plainly stated I didn't think you were commenting
| with ill intent. Ignorance doesn't require intent though.
|
| ASL isn't a "foreign" language. It's an American
| language. In fact, it's more "American" than English.
| People who communicate via ASL aren't foreigners on
| holiday. They're our friends and family and neighbors.
| You are absolutely correct in your assumption that being
| able to communicate with those around you is important!
|
| Imagine if everyone around you just refused to engage
| with you verbally and would only communicate via text
| messages. If you're speaking they completely ignore you
| until you write it down. If they're speaking while you're
| around it's always in whispers so that you can really
| only get bits and pieces. How would you feel? Annoyed?
| Excluded? Like you're not able to fully understand
| conversations?
|
| Your travel analogy doesn't hold water because (in
| addition to Deaf people not being foreigners or guests!)
| you seem unwilling or unable to understand that (1)
| hearing is a critical component of effective verbal
| communication and (2) Deaf people _can't_ hear. Again,
| I'll chalk it up to ignorance rather than ill intent but
| your analogy as a whole is pretty gross to paint Deaf
| people as entitled, unreasonable, and demanding. Your
| analogy is akin to suggesting it's unreasonable for me, a
| person who can walk, to rollerskate everywhere I go and
| it would be unreasonable for me to expect curb cuts just
| so I could rollerskate everywhere so therefore it's
| unreasonable for people who use a wheelchair to expect
| curb cuts. If someone who uses a wheelchair wants to use
| the sidewalk they should just try fucking _walking_
| right? I don't know why no one has thought of that!
| YurgenJurgensen wrote:
| There's nothing inherent to the world we have built that's
| stopping deaf people using the Internet in their own
| languages. Yet there's no ASL Wikipedia. Why not?
| elliekelly wrote:
| There are many languages that don't use a system of
| writing.
| YurgenJurgensen wrote:
| But there's not many that can't use a system of writing.
| phailhaus wrote:
| Unfortunately, there are very few Deaf people, so there is
| no world in which everyone learns ASL just in case they
| meet a Deaf person. It just doesn't scale, and so what you
| end up with is isolating them even further. The best
| approach is to bridge the gap: similar glasses can be used
| by hearing folks to translate sign language, and it
| theoretically generalizes to all foreign languages.
| elliekelly wrote:
| Check out that first link about Martha's Vineyard ;)
| phailhaus wrote:
| Yeah that's what I'm replying to. Most of the country is
| not like that so ASL is not going to be a shared
| language, and expecting them to move is not feasible
| either.
| gedy wrote:
| > Not that there's necessarily a technological way around those
| limitations either.
|
| I can see at some point here being able to wear AR glasses that
| overlay hand signing over the speaker
| norgie wrote:
| It is true and something more people should understand that ASL
| is not a signed version English, but most ASL speakers are
| pretty much bilingual. They are taught to read English, and
| most places also encourage learning to speak English, generally
| with speech-language pathologists, though some in the community
| are understandably reluctant.
| blamestross wrote:
| "most" is an interesting choice of word here.
|
| It's true. 70% of deaf children learn to read English [1].
| But I don't think that's enough to consider it a safe
| assumption.
|
| [1] https://www.handsandvoices.org/articles/education/advocac
| y/w...
| kzrdude wrote:
| Why isn't it more than 70%? I'm assuming this is in an
| english language majority language, so everyone that can
| should want to be able to read english.
| giraffe_lady wrote:
| Right, which is why I'm pointing out the second language
| nature of it since a lot of people are bilingual and I think
| have an intuitive grasp of the difference in ease between
| captions in your native language and captions in your second.
| metal_am wrote:
| Oral vs. sign is a big ongoing debate in the deaf education
| world. Different schools have different schools of thought.
| spullara wrote:
| Folks that use ASL use fingerspelling which is of course just
| written english, no?
|
| https://www.lifeprint.com/asl101/fingerspelling/fingerspelli...
| kiba wrote:
| No. Fingerspelled words are not written English, but
| literally spelled out English words.
|
| For example, it would be like me saying, "'h', 'e', 'l', 'l',
| 'o'" instead of whatever translated 'hello' in ASL.
| Tarragon wrote:
| Nope. https://www.signingsavvy.com/article/45/The+difference+
| betwe...
|
| Here's a fun example. ASL allows, maybe even requires,
| negation after the statement. An interpreter friend of mine
| was interpreting _Wayne 's World_ in a mixed crowd. The whole
| "<statement>... NOT!" joke gets laughs from the hearing
| audience and the Deaf audience doesn't understand why.
| giraffe_lady wrote:
| I have only basic ASL and am by no means an authority. But I
| think between native ASL speakers its use would be very rare,
| mostly just to clarify an exact spelling for something that
| was going to be written down in english.
|
| Native ASL speakers who are completely illiterate in english
| certainly exist, and I'm not sure at all if they know or use
| finger spelling.
| craigbaker wrote:
| Fingerspelling is just a path for borrowing individual words
| from English. It is not a part of native ASL vocabulary or
| grammar; that is to say, ASL does not consist of
| fingerspelled English sentences.
| [deleted]
| 1-6 wrote:
| Can they make the glasses without tint?
|
| I can imagine users going, "I'm deaf, not blind."
|
| Magic Leap has screens that can adjust opacity at the pixel-
| level.
| danscarfe wrote:
| The current generation of glasses (such as these Nreal Airs)
| use a birdbath technology which requires a tint. The next
| generation of waveguide glasses won't require this but they are
| currently not as good visual quality, especially for reading
| text. They are also a lot more money. Everything is a trade off
| right now. The next couple of years will be transformational.
| quantumquetzal wrote:
| Background: researched this space for a graduate degree.
|
| There are a few issues that are unanswered by this video (which
| isn't intended to be a technical deep dive, but I don't see any
| related links in the video description):
|
| 1. How do these glasses handle multiple simultaneous speakers?
| Based on the display I saw, it shows the speakers' words
| sequentially, which starts to fall apart in real-world
| environments, especially group conversations. This is a big
| problem, and wider adoption is contingent on handling this
| elegantly.
|
| 2. These appear to be the classic "smart glasses" display style
| that's pervasive in consumer head-worn displays today, where
| content is projected at a fixed depth in front of the wearer.
| Because the captions aren't anchored at the same focal distance
| as the speaker, the wearer's eyes will swap between the captions
| and the speaker's faces, which is a tiring activity, and can make
| the wearer feel like they're not part of the conversation or
| being rude.
|
| 3. As mentioned by another commenter, this is a useful idea for
| people who lose their hearing later in life. That said, this is
| less (although certainly still) useful for people who have
| congenital hearing loss and primarily communicate via ASL.
|
| All in all, it's exciting to see growing interest in this space,
| as it's easily extendable to people learning a new language or
| navigating a foreign country. I think offloading the speech-to-
| text to a tethered mobile device is a good choice (though it
| would be nice to do low-latency wireless transmission).
| justinator wrote:
| _> 3. As mentioned by another commenter, this is a useful idea
| for people who lose their hearing later in life. That said,
| this is less (although certainly still) useful for people who
| have congenital hearing loss and primarily communicate via
| ASL.>_
|
| Someone primarily communicates with ASL and then there's me
| that doesn't know ASL. I can speak to them, and they can read
| what I've spoken. That works pretty well. They communicate with
| me via text to speech, or (I guess in the near future) ASL to
| speech - however that will work.
|
| I mean, that's awesome.
| ape4 wrote:
| It would be nice if it put the captions over the speaker (in a
| speech bubble?)
| danscarfe wrote:
| You pin the captions next to the person. We did try speech
| bubbles, but it didn't look great, so we went back to simple
| subtitles.
| aeturnum wrote:
| This is a classic curbcut in the sense that it will help those
| with heading as much (if not more than) the D/deaf community.
| Still very excited for it - agree with all your questions and
| concerns.
|
| As usual, this marketing seems most directed at normate ideas
| about what disabled people want / need, but the tech seems very
| cool and there does seem to be potential. Without looking into
| the product deeply it seems like there are D/deaf people on the
| team, which gives me hope.
|
| I do wish that we would just embrace the idea that using
| machines to make information available in many mediums is
| something all people can use and appreciate.
| nohaydeprobleme wrote:
| To make the comment easier to read for others, it looks like
| the commenter may have made a slight typo and meant to write
| "hearing" instead of heading, to convey:
|
| > "This is a classic curbcut in the sense that it will help
| those with [edit: hearing] as much (if not more than)..."
| DrKeithDuggar wrote:
| Thank you for looking at XRAI Glass!
|
| 1. For multiple simultaneous speakers of comparable volume,
| it's only as good as the underlying speech-to-text engines
| we've implemented/integrated, which is currently not very good.
| It's active area of research and engineering for us and we
| believe we'll make strides to improve things; but, as you
| rightly point out, solving the crosstalk problem is very
| difficult. For the more general so-called "cocktail party
| problem", we can do a good job of filtering out more
| distant/lower volume voices and other environmental noise.
| Choosing the right microphone can improve things further, for
| example by pairing a noise canceling Bluetooth lapel mic.
|
| 2. We allow one to project the subtitles at varying depth,
| within the capabilities of glasses. We're seeing an effective
| focal depth range for fixed apparent size of about 0.5m to 3m.
| If one also allows change in apparent size, to simulate
| perspective scaling, the range is higher.
| roughly wrote:
| I imagine it'd substantially increase the compute load, but
| I'd be curious if you could use multiple microphones and beam
| forming to separate out the streams of speech and feed them
| to the TTS algorithm independently.
| kajecounterhack wrote:
| Single-mic source separation is possible in an unsupervised
| manner today that could probably work better than
| beamforming both compute-wise and with regard to
| implementation difficulty (you'd just need a lot of
| recordings to represent the space of sounds you want to
| separate).
| morcheeba wrote:
| Maybe a combination? Even simple beamforming/stereo would
| be helpful to help display the speaker's location. For
| example, the "speaker 1" tag could appear on the left,
| center, or right of the display to give a spatial clue
| where they are located.
| gumby wrote:
| I have a different use case, involving wearing them in the
| house: listen to what my girlfriend says, use some ML to
| analyze whether I need to know it and if so, put it on the
| display.
|
| She has a different "speech mode" than I: she speaks while I'm
| reading or washing the dishes or whatever, and sometimes it's
| to herself, sometimes to Siri, and sometimes it's something she
| wants me to know.
| cecilpl2 wrote:
| There is a simple wetware solution, which is that she learns
| to say your name before saying something she wants you to
| know.
| drjasonharrison wrote:
| Do you have any idea how hard it is to train partners to
| use your name once they fall out of practice? It's like
| there is a negative reinforcement stimulus applied
| everytime they do use your name.
|
| I noticed that my wife stopped using my name when it became
| more ambiguous as to who she was talking to because our
| kids are now older and conversations (really instructions
| and queries) are at the "adult content" level rather then
| "child content" level.
|
| Retraining her will be challenging.
| whitemary wrote:
| That would still require him to listen to his girlfriend
| talk. With all the technological innovation available
| today, nobody should have to listen to their girlfriends
| talk.
| sigwinch28 wrote:
| I hope (reatively immature) solutions like this will not be used
| as an excuse to remove accessible infrastructure from the world
| (e.g. captions at the cinema, live subtitles at the theatre, text
| displays on public transport).
| drjasonharrison wrote:
| There is a huge population group that I am hoping will demand
| and make accessibility much more refined. Everything from the
| size of text to lighting levels to subtitles
| sigwinch28 wrote:
| Which group is that? Disabled people, or their allies, or
| someone else?
| tossaway0 wrote:
| I admittedly don't have much experience with deaf people; I had
| one acquaintance in high school who was deaf. Hanging out with
| him made me very aware of how isolating it can be to only be able
| to participate in conversations where you are actively trying to
| pay attention.
|
| If these can let people hang out and participate without having
| to actively track each speaker in a group setting it will go a
| long way.
| allanrbo wrote:
| Bought a pair of these glasses, Nreal Air, a few months ago. I
| find them useful for laptop coding without straining my neck.
| It's awesome to see more creative use cases for them!
| Andrew_nenakhov wrote:
| Mod with speech bubbles would be high on my priorities list, and
| a bonus would be a comic book font.
| danscarfe wrote:
| Thanks for the shout-out! To try and answer your questions:
|
| 1) The cocktail party problem is still a WIP. This is a very hard
| problem to solve.
|
| 2) These are not 'viewer' glasses they are 3DOF glasses which
| support moving and pinning the subtitles in 3D space
|
| 3) Whilst we targeted the Deaf and HOF to begin, we see broad
| applicability beyond this
|
| 4) You don't need glasses to test it, just an Android 12+ phone.
| Download and try it. We'd love your feedback
| https://link.xrai.glass/app
| ogig wrote:
| Hi there: Some feedback from a sing language interpreter, my
| wife, as I showed her this.
|
| > 4) You don't need glasses to test it, just an Android 12+
| phone.
|
| This exactly pointed her. Some deaf people will use any
| dictation software on the phone, and look at the phone when
| needed. This glasses instead will cover some field of view.
| Note that view for deaf people is more important and used than
| for the rest of us. She couldn't see the improvement of using
| bulky glasses instead of lowering your eyes to the phone.
|
| Personally I think the endeavor is admirable and wish you best
| of luck. Also, as other comments say, I think this product
| might be more desirable for HOF and late in life hearing loss
| sectors than born deaf people.
| danscarfe wrote:
| The advantage of using AR glasses is you can still look at
| the person, see their facial expressions, the reactions etc
| without always having to look down at your phone. The glasses
| aren't very bulky or heavy. We're just providing an option
| for those that want it. It's a magical experience :)
| Blackthorn wrote:
| This is the sort of thing I expected Google Glass to be able to
| do. But Google apparently lacks sufficient imagination, so they
| just unceremoniously canned it. Really hope this takes off.
| dymk wrote:
| Google literally announced this feature 10 months ago at I/O
| 2022
| zoklet-enjoyer wrote:
| I was so excited for the possibilities of AR when I first heard
| about Google Glass. I imagined navigating foreign cities with
| signs auto-translated to English, turn by turn directions,
| translated subtitles, etc
| raldi wrote:
| I'm impressed at how not-awful the glasses look.
|
| I do wish there had been a real-life shot showing how the text
| appears to the wearer, though.
| danscarfe wrote:
| You can see how it appears in our promo video:
| https://www.youtube.com/watch?v=iilN0368vQU
| raldi wrote:
| The video gives the general idea, but it only shows a
| graphical simulation of what the wearer sees. I'm curious to
| see an actual photo or video of the text appearing on the
| lens of the glasses, in reality, with no special effects.
| Mizza wrote:
| Would love a version for translation. I don't need the glasses,
| just read the translation to me and I'll wear an ear bud. And if
| you could make it work at a loud party, that'd be just perfect..
| omoikane wrote:
| Looks like this product exists:
|
| https://support.google.com/googlepixelbuds/answer/7573100
| tsimionescu wrote:
| Speech to text barely works in the best conditions. Combine it
| with bad audio, automatic translation, and text to speech, and
| you'll be lucky if you understand 10% of what's being said.
| JoeAltmaier wrote:
| Other applications?
|
| Travelers, translate Spanish to English etc.
|
| Reconstruct voice in noisy environments.
|
| ???
| danscarfe wrote:
| We support both!
| gus_massa wrote:
| In Spanish to English are you doing a word by word
| translation, r fixing some basic gramar (for example changing
| the order of adjectives and nouns)?
| elil17 wrote:
| I'm a hearing person and I've spent a summer interning in a 50/50
| mixed Deaf and hearing research group.
|
| My take is that this is a huge UI improvement for AI speech to
| text, which a lot of Deaf people are already using to listen to
| conversations. It seems particularity great because it allows
| this technology to provide situational awareness while, for
| example, walking.
|
| It's important to remember, though that, for conversations where
| you're trying to include a Deaf person who isn't good at speaking
| or chooses not to speak, speech to text is a fundamentally
| unequal communication modality. They will be able to "receive"
| but they won't be able to "transmit", which makes for extremely
| lopsided conversations. There is no substitute for taking the
| time to learn a sign language or to have conversations via
| writing (no substitute for sign language as it requires a lot of
| patience from both parties).
| jackblemming wrote:
| Why can't they also make glasses that translate sign language
| into English audio or text then?
| dgunay wrote:
| Not an expert in either technology but I would imagine a lot
| of the difficulties of speech to text (bad recording
| conditions, variety of accents & pronunciation differences,
| etc) also have analogues in computer recognition of hand
| signs (camera alignment is bad, cut off, lighting is bad,
| someone's hand signs are lazily performed or slightly
| different than textbook ASL, etc).
|
| Speech to text was "solved" a long time ago but I've seen it
| take many years to become as usable as it has recently. And
| it still regularly is frustrating to use for me!
| tsimionescu wrote:
| My experience of speech to text has been terrible. As soon
| as someone has a bit of an accent, it becomes completely
| incoherent, even if all participants understand the accent
| perfectly. Even for good accents, using any kind of
| technical terms or proper names throws it off. And even
| when none of these are a problem, it still has at least a
| few percent error, even for English.
| dymk wrote:
| I'm sure that's something someone could do! And it sounds
| like a very fun project. It probably hasn't already been done
| because there already exist very good speech-to-text models
| and not many (any?) sign-language-to-{text,audio} models
| zie wrote:
| Sign language is a complex language, so something has to
| learn yet another language. Perhaps made all the more complex
| as it's spatial and visual instead of verbal or written.
|
| Not every deaf person uses sign language and there are many
| different sign languages in the world. American Sign
| Language(ASL) is but one of these languages.
| richbell wrote:
| > Sign language is a complex language
|
| Specifically, sign languages are not visual representations
| of existing languages (e.g. ASL and English) but completely
| different languages altogether.
| zie wrote:
| Exactly. Though Visual "representations"(for lack of a
| better word) of English do exist in dialects like
| SEE(Signed Exact English), which is obviously not the
| same as ASL.
|
| I'm sure other signed languages have rough equivalents in
| their regional areas as well. I know Mexico has a few
| different signed languages, though I only have passing
| familiarity with 1 of their signed languages and it's
| definitely not a representation of Spanish.
| SamuelAdams wrote:
| And even then, there are different dialects of ASL.
| hinkley wrote:
| We get this wrong even for hearing people in a lot of
| situations. Conference calls in particular, the people in the
| room have a different experience from those on the call. Side
| conversations in a single room can be disruptive, but over a
| phone the secondary conversation and the primary can turn into
| an unintelligible mash.
|
| Hard of hearing people have the same problem in person. We
| aren't really at a place yet where someone wearing hearing aids
| still has 3d hearing. So like on a conference call, they can't
| figure out what's going on when four people are talking at
| once.
|
| I know a partially deaf kid who prefers socializing on Discord,
| because of this. Everyone is equal and all conversations have
| to be 2 people at a time.
___________________________________________________________________
(page generated 2023-03-14 23:01 UTC)