[HN Gopher] Hands-Free Coding: How I develop software using dict...
___________________________________________________________________
Hands-Free Coding: How I develop software using dictation and eye-
tracking
Author : joshwcomeau
Score : 435 points
Date : 2020-10-21 11:53 UTC (11 hours ago)
(HTM) web link (joshwcomeau.com)
(TXT) w3m dump (joshwcomeau.com)
| azinman2 wrote:
| For RSI folks, my experience is that RSI can "move" to other
| parts of the body pretty easily, including eyes and vocal chords.
| Somehow doing "work" with a part of the body can strain it, even
| when talking all day long socially wouldn't.
| reanimus wrote:
| This is really, really interesting -- I've often wondered about
| how the accessibility space is meeting the needs of disabled
| programmers, especially after I briefly lost my vision due to an
| eye injury.
|
| The eye-tracking setup in particular looks really nice. Even if I
| stuck to a regular keyboard flow I feel like that would be
| incredibly useful in reducing strain.
| heinrichhartman wrote:
| > The biggest issue I've found so far is voice strain
|
| My singing teacher started out as a voice therapist. He is still
| working with a lot of people with damaged voices that are there
| for therapeutic reasons as well as professional opera singers
| that there to level-up their game.
|
| He says, the best thing you can do for your voice is to sing.
|
| Singing will give you the proper "support" from your lungs and
| train the vocal muscles without wearing them out. One of the key
| aspects here is producing high quality vowels, and don't let the
| sound be disturbed by consonants. This is opposite to how the
| voice is used when speaking (in most languages), where consonants
| are of primary importance for people to understand what you are
| saying. However, consonants are interruptions in the air-flow and
| can easily wear the voice out and cause stain.
| joshwcomeau wrote:
| Interesting! Maybe I'll look into a voice trainer.
| rohith2506 wrote:
| I have suffered with thoracic outlet syndrome for almost 2 years.
| Countless physical therapy and deep tissue massages helped
| alleviate the pain but nothing seems to solve the root cause. I
| am 26 and the thought of not being able to use my hands to do
| something I have frightened the hell out of me.
|
| That's when I stumbled upon John Sarno and his book "The mindbody
| prescription". I thought it's pseudoscience and I read it with
| very low expectations. And It's been almost four months since I
| have trouble using my computer.
|
| In a nutshell, most of RSI injuries are not just physical
| problems and they are tightly coupled with your mind. The most
| important step is to acknowledge that subconscious thoughts
| related to pain and work along with it. Whenever I feel a
| tingling sensation, I yell at myself ( In my head ) that it's all
| in my mind and I have started to feel normal. Of course, this
| might not work for everyone but definitely worth a try.
|
| And I also do lot of strength training and climbing which really
| helped strengthening my hands
| rsa25519 wrote:
| I'm not so sure about this. My understanding is that RSI is
| caused by inflammation, and ignoring it will only make it
| worse, to the point of needing surgery. Although it sounds like
| you've been dealing with this longer, so I'll give you the
| benefit of the doubt
| rohith2506 wrote:
| I see where you are coming from and I strongly suggest to try
| the conventional route to eliminate any serious physical
| problems before ending up here. My point is that RSI problems
| are not just physical but tightly coupled up with your mind.
| I wish someone can do a better job of explaining
| neuromuscular connection such that this won't be
| pseudoscience anymore.
| fao_ wrote:
| > In English, an ordinal number is one used to describe order,
| like "fifth" or "ninth" or "three hundredth". In Talon, they're
| used to repeat commands. If I wanted to go left by 9 spaces, I
| would say go left ninth.
|
| Right! Like 9l in vim, but swapped.
|
| I've always thought vim-like interfaces would be a good way to
| structure vocal input and I'm glad there are solutions operating
| in that space.
|
| Edit: beaten to the punch :)
| rco8786 wrote:
| The eye tracking sounds like it could be incredibly useful as a
| 3rd input.
|
| Very cool stuff, thanks for sharing
| Ottolay wrote:
| In addition to a pointer replacement, there is an idea of using
| eye tracking on a virtual keyboard with a swype type entry.
| haxiomic wrote:
| That's a great idea! This would really push up input speed.
| I'm surprised I haven't seen any implementations in the wild
| yet, although research has been done to explore this [0]
|
| [0] https://pdfs.semanticscholar.org/1170/44feb247a82ab81a30b
| 57f...
| weeb wrote:
| Windows 10's Eye Control has something like this, called
| "Shape Writing". See the second GIF here:
| https://www.theverge.com/2017/8/2/16087368/microsoft-eye-
| con...
|
| Long before that, Optikey has had "multi-key selection"
| which works in the same way, see
| https://youtu.be/HLkyORh7vKk?t=10
|
| In my experience (as a developer of eye gaze interfaces,
| not an everyday end user) it is often more efficient to
| have really good next-word-prediction (such as with the
| Presage engine) combined with single-letter dwells, rather
| than using "swipe-like" spelling, where you're committed to
| tracing out the whole word.
| charris000 wrote:
| I dunno... "swyping" your eyes around a virtual keyboard
| seems like it'd cause a lot of strain on your eyes.
| falcolas wrote:
| There are alternative virtual keyboard styles (typically
| made for use with a mouse and impaired motor control) which
| would work better. Fewer "fine" movements.
|
| Combined with predictive typing, and you could go pretty
| far.
| Ventilatorr wrote:
| Do you have any examples?
| weeb wrote:
| Not the OP, but Dasher is a classic example that works
| very well for a single 2D input like using a joystick:
| https://youtu.be/0d6yIquOKQ0
| thesuitonym wrote:
| I don't know about that, but I'd love to have eye tracking
| for focusing windows. Since I don't generally maximize
| windows, I find that I'm often typing hotkeys for the window
| I'm looking at, not the window my mouse is on (or was last
| focused, depending on your wm)
| baxtr wrote:
| Reminds me of the Figma GPT-3 Plugin
|
| https://mobile.twitter.com/edleonklinger/status/128425141917...
| vectorphresh wrote:
| I wonder if the author has considered integrating his voice
| interface with an AI resource like GPT-3. I could see future
| development being done in this fashion.
| HumanMusic wrote:
| Right ear-free audio
| goldforever wrote:
| Useless
| psychometry wrote:
| Seems like it would be a useful feature in voice recognition to
| have a "whisper" mode to prevent strain.
| Karawebnetwork wrote:
| There are some high tech breakthrough in this space. You can
| have microphones that hear the faintest of internal voices and
| even non-invasive ways to hook on the nerves to get your
| internal monologue.
|
| https://www.smithsonianmag.com/innovation/device-can-hear-vo...
| quicklime wrote:
| I once had laryngitis for about a week, and during this time
| found out that for most people whispering puts more strain on
| the vocal cords than regular speaking:
| https://www.nytimes.com/2011/02/08/health/08really.html
|
| But a whisper mode might prevent ear strain for those working
| around you :)
| thecrumb wrote:
| I also bookmarked https://serenade.ai/ after seeing it here on
| HN. 10 years ago I shattered my hand and collarbone and used
| Dragon which sort of worked but was not specific to coding. These
| new tools look really neat.
| polyterative wrote:
| as a dev with severe hand pain I would love more tools and help
| like this
| goldforever wrote:
| All this to build a web app? Useless!
| tomcam wrote:
| The demo is incredible and heartening. I did not know the state
| of the art had improved so much. Thank you for sharing this,
| Josh.
| melling wrote:
| The author links to this other project, which I've never heard
| of:
|
| https://serenade.ai/
|
| He also references Tavis Rudd's viral voice coding video, which
| is already 7 years old.
|
| https://www.youtube.com/watch?v=8SkdfdXWYaI
|
| The future is arriving slower than i would have guessed. I
| thought we'd be developing on the iPad using voice, gestures, and
| eye tracking by now.
|
| Can I a least get "build and run" with my voice soon?
| CoffeePython wrote:
| I'm very surprised none of the videos on the serenade landing
| page have audio.
|
| I would love to see/hear what it's like. I'd suspect adding
| audio to those videos would be a win on conversions.
| tmacwilliam wrote:
| Thanks for the suggestion! We recently revamped our homepage,
| and we're in the process of recording some new videos. Will
| keep that in mind!
|
| In the meantime, here's a video that a developer using
| Serenade recently made about their first experience!
| https://www.youtube.com/watch?v=Pc-EbY1fRWk
| guram11 wrote:
| hands free and perl out loud here
| https://www.youtube.com/watch?v=Mz3JeYfBTcY
| chrismorgan wrote:
| > _I 've heard that learning Vim can make this much more
| effective._
|
| This doesn't surprise me; Talon is a lot like Vim.
|
| They're both modal: Talon's command mode roughly matches Vim's
| normal mode, and its dictation mode roughly matches Vim's insert
| mode.
|
| Talon's ordinals apply to Vim operations (normal mode) as well:
| operations can be preceded by a number which normally says how
| many times to do it. Talon's "go left ninth" would be 9h or 9- in
| Vim. (h goes left one character.) "One zero third" is more
| cumbersome because in Vim it entails switching between modes
| (Talon and Vim modes don't match precisely), but starting in
| normal mode, a1<Esc>3a0<Esc> works to append 1000 where the
| cursor is. (a to enter insert mode after the current position, 1
| to type a 1, Escape to return to normal mode, 3 to say "do the
| next thing three times", the next thing being appending a zero.)
|
| Both Talon and Vim value this sort of consistency in being able
| to build bigger things from smaller pieces.
|
| So I imagine Talon would work quite nicely with Vim, once you get
| talon to shift out of the way in some cases and let Vim handle
| those things. (It's similar with browser extensions that let you
| interact with the browser Vim-style: when the page in
| consideration has extensive key bindings, especially Vim-like
| things like j/k for down/up, the extension actually gets in the
| way.) I expect you'd find that having used one made learning the
| other a good deal easier, since they're working in just the same
| ways.
| lunixbochs wrote:
| I developed much of the original Talon code using a single
| Python file (the combination of std.py + basic_keys.py) and
| Vim.
| expjpi wrote:
| > Honestly, it's just been such a relief to discover that my
| hands aren't needed for me to do my work.
|
| This describes my reaction exactly. I'm noticing my rsi getting
| better, which I'm grateful for, but the gift of not being so
| scared feels like the bigger deal.
| SimonPStevens wrote:
| The eye tracking looks great, just as a regular additional way of
| using the mouse for things like focusing windows or tapping
| dialogs. I'm very tempted to get one of those.
|
| Does anyone with experience of them know what the multi-monitor
| support is like? I have 3 screens side be side and one below, so
| quite a large range to cover that requires some head movement to
| go across all of them. Will it work with viewing over that wide
| of a space where I'm not always directly facing the primary
| screen?
| bwaine wrote:
| I don't know why, but I've often thought about the situation the
| Josh finds himself in. IE - having to find a different way of
| building software if I couldn't use a keyboard and mouse due to
| medical condition.
|
| The solution I always imagine is paying someone or having my
| employer pay someone to strong style pair program with me.
| Perhaps a student, junior developer or even someone unfamiliar
| with software development entirely.
|
| For those unfamiliar with strong style, this rule sums it up:
| "For an idea to go from your head into the computer it MUST go
| through someone else's hands". Like the standard driver /
| navigator pair programming technique but with the navigator never
| touching the keyboard.
|
| In the case of someone completely unfamiliar with software
| development I imagine that there would initially be a dramatic
| high / low skill gradient between us, with the person essentially
| transcribing. However given the intensity of the practise I think
| this gradient would level out quite quickly.
| GordonS wrote:
| > The solution I always imagine is paying someone or having my
| employer pay someone to strong style pair program with me
|
| Wow, that's some impressive out-of-the-box thinking!
|
| The OP's situation is something I worry about, as I'm worried
| my neuropathy will reach a point where I can't use my arms to
| type/mouse all day. I spent some time researching speech input
| for coding a year ago, and found a few fledgling solutions, but
| I'd need to expend a _lot_ of time to get something remotely
| workable.
|
| Your idea is, if I may say so, _genius_ , and having that as a
| backup sets my mind greatly at ease, so thanks.
| bwaine wrote:
| I'm sorry to hear about your neuropathy, I'm glad my
| suggestion eases your mind!
|
| On pairing in general: For a long period of time when I
| worked at bigger companies (before co-founding my current
| venture) I pair programmed 90% about of the time.
|
| For the most part I really enjoyed it, we alternated pairs
| quite frequently and I worked with people at a range of
| seniorities. I really enjoyed alternating between playing the
| role of teacher vs learning from someone much better at the
| (insert programming language, database, business domain).
|
| I think during that time I also had some of the most
| stimulating and deep conversations about software
| development. When pairing, pairs have a deep shared context
| that's hard to replicate in other scenarios.
|
| There are some drawbacks though, the main one being it's
| extremely full on and emotionally tiring. I'd class myself as
| a high functioning introvert and after a day of pairing I
| needed some serious regenerative quiet time!
|
| My advice to someone considering this suggestion would be to
| try some pairing now with friends or colleagues and get a
| feel for it and see if it works for your personality and
| working style.
| GordonS wrote:
| > I'd class myself as a high functioning introvert
|
| Hah, I'm stealing that phrase to describe myself too!
|
| I've never "formally" done pair programming, but have sat
| and coded/reviewed with colleagues for short stints, and
| generally found it OK.
|
| From time to time I have to do full days (or several days)
| of workshops, and find those _really_ tiring and
| emotionally draining. I always think I fare better 1-1 with
| people though, so if pairing was a necessity, I 'm sure I
| could adapt.
| ianhorn wrote:
| I've been having similar issues lately, and after looking at
| talon, serenade, and caster, I ended up using caster [0]. The
| different programs all have significant differences in usability,
| and have clever ideas behind them, but unfortunately automatic
| speech recognition is still bad enough that the primary factor is
| which has a better ASR engine. Caster supports Dragon Naturally
| Speaking, which is expensive, but enough better to make it
| worthwhile.
|
| There are moments where I think this is going to be the future of
| programming, since code as text is only the easiest for as long
| as typing is the easiest way to record things unambiguously.
|
| But for the most part it's still pretty frustrating. If ASR
| systems can get the sentence error rate down by an order of
| magnitude or two, I sincerely think this will take off not just
| for accessibility, but for normal use.
|
| Until then, it's a PITA that is saving my career nonetheless.
|
| [0] https://caster.readthedocs.io/en/latest/
| cybwraith wrote:
| Do you know of any workflows like this using entirely open
| source software?
|
| EDIT: Seems like caster itself has instructions for an open
| source recognition engine to pair with it. Not sure how
| accurate it'll be but I'm going to give it a shot!
|
| I hate the idea of relying on closed source software to be able
| to continue in my profession. If this works I'm definitely
| going to be donating to the FOSS options
| whimsicalism wrote:
| My understanding is that Talon is built off of wav2letter's
| inference framework for ASR.
| lunixbochs wrote:
| I do not use the wav2letter@anywhere inference frontend - I
| trained the acoustic model using the facebook upstream
| code, but the decoder is almost entirely new, and on
| Windows I use Pytorch for inference.
|
| Talon ships with a libw2l.so/dylib on Linux/Mac built from
| my open source repos.
| ianhorn wrote:
| Yeah there is the Kaldi backend for caster, which I've tried
| on my Mac machine, since Dragon isn't a thing on Mac.
| Unfortunately it's not nearly as good :-(
|
| I'd like to try to record my usage of Dragon that I could
| fine-tune my own model, but it's harder to get around to
| hobby coding projects like that now that coding is more of a
| pain in the ass.
|
| The unfortunate reality in my case is that using Dragon is
| the least frustrating way to keep working. I don't think
| closed paid ASR models will stop being noticeably better
| better than three open ones until the state-of-the-art error
| rate is basically zero.
|
| I do like that caster is open source where talon isn't, but
| preferences like that are pretty low priority when say your
| career is on the line.
| lunixbochs wrote:
| Fwiw talon has a setting to record everything locally,
| annotated with the recognized words. In the next release it
| will also work when using dragon.
| rntz wrote:
| Talon actually can work with Dragon; it started out requiring
| Dragon but has since grown its own voice recognition.
| Anecdotally, a lot of folks on the talon slack seem to be
| refugees from Dragon-based solutions, and a common observation
| is that Talon's voice recognition is much lower-latency than
| Dragon. Unfortunately I can't confirm this as I haven't used
| Dragon myself.
|
| My impression of talon's speech recognition is that it's good
| enough for voice coding, but could be improved when it comes to
| dictating English prose. That said, there are promising avenues
| of improvement. If you pay for the Talon beta, there's a more
| advanced voice recognition engine available that's much better
| at English, and you can also hook into Chrome's voice
| recognition for dictation.
| lunixbochs wrote:
| > Talon actually can work with Dragon
|
| More specifically, "can work" means: if you run Talon on
| Windows or Mac alongside Dragon, and don't configure Talon to
| use its own engine, Talon will automatically use Dragon as
| the recognition engine.
| ianhorn wrote:
| > or Mac alongside Dragon
|
| Unfortunately it looks like Dragon on Mac isn't a thing
| anymore, which is a shame because my main dev machine is a
| Mac. Or at least it used to be, before I needed to start
| dictating everything.
| lunixbochs wrote:
| Talon can also use Dragon remotely from a Windows machine
| or VM, kind of like aenea but more tightly integrated
| (the host is driving instead of the guest).
|
| And I do have some users voluntarily switching from Mac
| Dragon to the improved speech engine in the Talon beta.
| Mac Dragon has been kind of buggy for the last few years
| so you're not missing much.
| ianhorn wrote:
| Any chance you have pointers on how to set that up? You'd
| probably laugh/cry to see my setup right now, with my
| Windows desktop on the left monitors and my MacBook on
| the right monitors because I need both... purely because
| Dragon is only sold on Windows since this started been an
| issue for me. A more tightly coupled super-aenea sounds
| pretty fantastic.
| lunixbochs wrote:
| Sure: First run Talon on both sides. Then go to your
| talon home directory (click talon icon in tray ->
| scripting -> open ~/talon). There's a draconity.toml file
| there.
|
| On the Dragon side, you need to uncomment and update the
| `[[socket]]` section to listen on an accessible IP.
|
| On the client side (Mac in your case), uncomment / update
| the `[[remote]]` section to point at your other machine.
|
| You also need to make sure both configs have the same
| secret value.
|
| From there, restart Dragon and look in the Talon log on
| the Mac side for a line like "activating speech engine:
| Dragon".
|
| To prevent command conflicts, I recommend setting Dragon
| to "command mode" (if you have DPI), and only adding
| scripts to Talon on the Mac side.
|
| If it doesn't work, you can uncomment the `logfile` line
| in draconity.toml on the Dragon side, restart Dragon, and
| look in the log to see what's going on.
| codenesium wrote:
| I personally have had carpal tunnel release surgery and bilateral
| rib resection for thoracic outlet syndrome. These surgeries fixed
| most of my issues but for the last couple of years I haven't
| tolerated sitting well because of my back so I use a standing
| desk. I've thought about a computer interface that is the
| opposite of this article. Something that requires more non-
| repetitive physical movement so that you're exercising while you
| program. Sure it would take longer to type but we all know
| working on a computer all day is bad for our health.
| joeseppy wrote:
| Sounds like an interesting idea for a VR/AR game. I'm picturing
| somewhere between fruit ninja and a vana white simulator
| pixelbreaker wrote:
| Redox keyboard is amazing, never had any pain since starting to
| use one about 18 months ago.
| mncharity wrote:
| Exploratory building of rich non-traditional (not necessarily
| handless) user interfaces is becoming increasingly accessible.
| For instance, here's a web demo of Google's MediaPipe's face and
| iris tracking[1]. And hand tracking[2] with a downward-facing
| camera, permits enriching keyboard and touchpad/tablets:
| annotating events with which finger was used, and where on a
| keycap it was pressed; hand and finger gestures in 2D and 3D; and
| positional touchless events. And speech to text... sigh.
|
| But doing sensor fusion is hard. And strongly impacts system
| architecture. "Launch the missiles"... 1000 ms later... oh, nope,
| that was " _Lunch is mussels_ in butter ". "Spacebar keypress
| event"... 50 ms later... "btw, that was a thumb, at the 20%
| position". "Ok, so 2000 ms ago, just before the foo, there was a
| 3D gesture bar". So app state needs to easily roll backward and
| forward, because you won't fully know what happened now until
| seconds from now. Upside is traditional "have to wait a bit until
| we know whether mumble" latencies can be optimistically
| speculated away.
|
| [1] https://viz.mediapipe.dev/demo/iris_tracking ("run" button is
| top right) [2] https://viz.mediapipe.dev/demo/hand_tracking
| Symbiote wrote:
| When I started my undergraduate degree, there was a PhD student
| everyone knew -- he looked like he ran the Linux User Group,
| which he did, he coded his own window manager, wrote games in
| Haskell etc.
|
| At some point, he got some hand injury that meant he couldn't
| type normally for months. He would only have been 21 or 22.
|
| This scared me, so since then I've followed the standard advice
| on avoiding computer-related injuries very carefully. I use a
| desktop computer with an external, adjustable monitor. I use
| ergonomic keyboards, and use a mouse with either hand. I learned
| to touch-type with the Dvorak layout, I have an adjustable desk,
| etc.
|
| I haven't had any problems, and I don't know which (if any) of
| these actions helped, but I'm surprised when I see other
| developers apparently happy to hunch over a laptop keyboard for 8
| hours, while sitting on an awkward chair in a coffee shop. Why do
| this to yourself?
| ch4s3 wrote:
| Especially the macbook pro keyboards. Those really hurt my
| hands.
| cschneid wrote:
| A few years back I started buying disability insurance, in
| addition to paying more attention to my ergonomics.
|
| My greatest economic asset is the years left in my career. For
| ~$1500/year I insure that. It's not cheap, but it would pay out
| until I was 65 at some reasonable percentage of my normal
| salary in the case I was severely injured, or otherwise unable
| to work. Anywhere from an ergonomics caused hand/arm injury, a
| bike-riding caused head injury, a wood-shop caused eye damage,
| etc.
|
| Consider grabbing it while you're young and working in a
| lucrative career.
| melling wrote:
| Not sure how old you are but I imagine over the next couple
| of decades we'll eventually get more advanced computer
| interfaces.
|
| Gestures, voice, eye tracking, ...
|
| Being a software developer could be a job for those who
| physically can't do other jobs.
|
| Subtle input gestures with Soli would be interesting.
|
| https://atap.google.com/soli/
| cschneid wrote:
| Assuming physical injury (like RSI or trauma), yes, it's
| getting easier and easier.
|
| But there are many other ways you could be injured and
| unable to work. My straw-man injury when I bought the
| policy was: "I'm a new skier, which is famous for major
| head trauma". Similarly, cancer, which directly or via
| treatment, causes exhaustion is another example of
| disability that is not solved with "if only I could just
| input text easier".
|
| I'm glad there's so much work in accessibility going on,
| and that hardware, machine learning, and social awareness
| has gotten widespread enough to let non-huge companies dive
| in and make real progress. It's important to open the field
| to as many people who we can.
|
| I just want to be sure people don't rely on technology
| being advanced enough at the exact time you may need it.
| You can easily get unlucky, and insurance exists to pay
| somebody else to take the risk.
| melling wrote:
| Yes, that makes sense. However, the older I get the more
| I think society should provide some sort of safety net
| for the unlucky.
|
| It's inefficient for everyone to do this and many cannot.
|
| The premiums could jump 50% in a decade, for example.
| I've known situations somewhat like yours where premiums
| went up a lot.
| tomcam wrote:
| I have a similar story. When I was in college I got a
| repetitive stress injury from practicing piano too much. I
| ended up becoming a programmer but I have followed ergonomic
| practices religiously in the 40 years since then. My kids make
| affectionate fun of my perfect posture. But I have worked with
| the computer keyboard for going on five decades and have never
| come close to suffering a repetitive stress injury.
| hnrodey wrote:
| I think ..... most people have no idea what they actually look
| like when hunched over a keyboard. It would do wonders to have
| a large mirror next to our workspace so one could see just how
| preposterous they're contorting their body.
| jedimastert wrote:
| I had a bit of a scare around the same time. I was following
| dual passions of computer science and piano playing, meaning
| around 12 hours a day of typing and/or piano-ing. I started to
| develop regular tendinitis in my wrists and fell down a rabbit
| hole of ergonomics and posture. With computers you can replace
| keyboards if you have money, but I was a musician in college.
| Also, you can't really change a piano keyboard all that much
| (and you really wouldn't want to) so I did a real deep dive
| into posture and getting the most out of what I already had
| luckylion wrote:
| I love the eye tracking. Even without having any issues with my
| hands, I'd like to have quick mouse movements without having to
| take a hand off of the key board. Zooming in to have more
| precision is clever.
|
| It seems the current generation systems aren't working for Linux
| yet, but that gives me hope to have a way to work with it at some
| point in the future.
| lunixbochs wrote:
| > It seems the current generation systems aren't working for
| Linux yet
|
| This isn't correct. Talon supports eye tracking on all of
| Windows, Linux, and Mac with the same hardware.
| ornornor wrote:
| I think parent is talking about the Tobii. I was also very
| very excited about it (I can't find a mouse or trackball that
| won't give me hand or shoulder or wrist pain) but then I saw
| it only works on windows... and nothing else. No Linux
| support. So that's a no go.
|
| If someone knows of such a device for Linux, I'd love to hear
| about it!
| lunixbochs wrote:
| Let me be more specific. The Tobii 4c, PCEye Mini, and
| Tobii 5* work approximately the same on Linux, Mac, and
| Windows with Talon out of the box.
|
| * The Tobii 5 does work, but is a work in progress and has
| some caveats until I do more work on it.
| luckylion wrote:
| Oh, that's great news. I thought Talon needed the drivers to
| work and was constrained in that regard. I will need to free
| up some time and try it out.
| JabavuAdams wrote:
| Thanks for this! Sorry it had to come about as a result of
| injury/disease. Hopefully you're not going to hurt your vocal
| tract -- are there precautions you take against this?
|
| My interest currently is in voice-assistants and developing
| something like a personal assistant / code-monkey / grad-student.
|
| I did have a stroke a few years ago, from which I've recovered,
| but it made me realize that almost all of my work, hobbies,
| aspirations, are tied to fairly extensive hand-eye integration.
| It's good to start building a Plan B before I need it.
|
| On a(nother?) side-note, my 88 year old dad still codes and his
| experiences have really shown me how bad we are at accessibility,
| as a developer community.
|
| One final thing -- watching you work in this way highlights for
| me how low-level a lot of the stuff we do as developer is.
| There's a certain amount of just text massaging that seems
| irrelevant to what we're actually trying to get done (although it
| can be fun/soothing/aesthetic).
|
| EDIT> The last point is why I can't give a fark about
| Vim/Emacs/whatever debates and obsessing. It seems like it might
| be fertile ground, but IMNSHO it's a trap.
| mncharity wrote:
| Tavis Rudd, "Using Python to Code by Voice", 2013:
| https://www.youtube.com/watch?v=8SkdfdXWYaI . Old but fun.
| ateng wrote:
| As a developer who also has cubital tunnel syndrome, having
| keyboards that places commonly use modifier keys [0] on your
| thumbs, such as Kinesis advantage or Ergodox, helps reducing the
| fatigue and symptoms immensely.
|
| [0] it is called Emacs pinkies for a reason, and yes, my CTS is
| totally emacs's fault as well.
| noir_lord wrote:
| Had pain a few years ago, switched to Ergo 4000's everywhere
| and have never had any pain since - depending on desk height
| I'll use the front riser but generally don't need it.
|
| I love them so much I have spare new-in-box ones stashed as a
| hedge for the day they stop selling them.
|
| They are about the cheapest good quality ergo keyboard I could
| find.
| ihaveajob wrote:
| It's pretty telling that RMS had to stop coding because CTS.
|
| Edit: Not carpal tunnel syndrome, as he explains in
| https://stallman.org/stallman-computing.html
|
| """In the mid 90s I had bad hand pain, so bad that most of the
| day I could only type with one finger. The FSF hired typists
| for me part of the day, and part of the day I tolerated the
| pain. After a few years I found out that this was due to the
| hard keys of my keyboard. I switched to a keyboard with lighter
| key pressure and the problem mostly went away.
|
| My problem was not carpal tunnel syndrome: I avoid that by
| keeping my wrists pretty straight as I type. There are several
| kinds of hand injuries that can be caused by repetitive stress;
| don't assume you have the one you heard of."""
| crispyambulance wrote:
| > The FSF hired typists for me part of the day...
|
| That must have been a job from hell, probably went through a
| lot of them!
| falcolas wrote:
| Manual transcription is still a thing today. Stenographers
| being one of the more well-known examples.
| zozbot234 wrote:
| Stenotype keyboards are way better from a RSI point of
| view, though. Unfortunately, no one has come up with the
| "theories" or systems that would allow use of them for
| specialized input like programming.
| xyzzy_plugh wrote:
| Not to flame-bait, but this is actually one of the reasons I've
| doubled down on vim. I like that I can function perfectly fine
| with one finger, I even managed to write a bunch of code using
| vim on my phone, and was surprised how well it translated.
|
| You could definitely set up toggled modifier keys and do the
| same with emacs, FWIW.
| Quarrelsome wrote:
| ye, modifier keys suck. This is why I prefer snake-case over
| the other casings because it needs no modifier key.
| fctorial wrote:
| What languages do you use?
| adamnew123456 wrote:
| Also along those lines: what keyboard layout? I've never
| seen a (QWERTY) layout where underscore isn't a shift-level
| key.
| 867_5309_Jenny wrote:
| Switching to a planck keyboard would let you customize all the
| positions of buttons for less strain.
| Symbiote wrote:
| The ErgoDox already allows arbitrary repositioning of the
| keys.
|
| The Planck keyboard is cramped. It's worse than a cheap
| Microsoft ergonomic keyboard.
| irjustin wrote:
| The planck keyboard is a great keyboard, but possibly on par
| with the worst in terms of ergonomics.
|
| Ergodox, Diverge, Keyboard.io, really any split key keyboard
| does worlds for keeping hands straight and are fully
| programmable to very high degrees.
| T-hawk wrote:
| Am I the only one who just keeps their hands/wrists
| straight on a regular unsplit keyboard? I've always
| naturally done this and never had any problems with strain.
|
| The best way to describe it is that the home keys aren't
| ASDF, but more like WEFV with my forearm angled so it's
| straight from elbow to fingertip. This exaggerates the
| effect but demonstrates it clearly; the real home positions
| are more like intermediate points between keys.
| aleksei wrote:
| Yeah, same. I find keeping my fingers on the home keys
| makes me twist my wrists outwards, which gets
| uncomfortable pretty fast. I mostly try to minimise wrist
| action in general.
|
| I've actually had a Planck EZ for a few weeks now and I
| agree it's probably not very ergonomic for the classic
| qwerty touch typist, or at least not much better than a
| normal keyboard barring the programmability.
|
| But since I don't do that anyway I find the keyboard to
| be pretty nice in terms of customisability and avoiding
| stretching.
|
| In general I feel my hands are used most naturally in
| close proximity to each other (at roughly abdomen height)
| so I'm drawn to small keyboards with lots of modifier
| keys. A spherical keyboard would be pretty interesting to
| try out.
| OJFord wrote:
| On my keyboard - which is.. I don't want to say 'full
| size', because maybe it's '80%' or whatever, but it's a
| regular external keyboard with numpad - f & j are centre
| to centre 60mm apart.
|
| If I outstretch my arms, straight, my index fingers are
| pretty much in line with my shoulders.
|
| So no, when I use my regular unsplit keyboard my
| hands/wrists are not straight, because my shoulders are
| not 60mm apart..? Am I missing something?
| estsauver wrote:
| I have a planck keyboard, but I think it's small size isn't
| fantastic for hand position. I think I may eventually
| purchase an ergodox ez to get better hand positions.
| Snitch-Thursday wrote:
| I've been wanting something like Talon for years. Easily
| customized voice recognition for commands (you hear me say
| 'blah', you type/start a program/click a button labelled 'blah')
| is something I've not had since I was very young and tried to
| trick out LCARS x32 using only Windows XPs speech recognition
| software using the crappy stick mics that lived on top of every
| single beige CRT monitor at school.
| 2Gkashmiri wrote:
| I have had an issue with dictation software... I want a dictation
| software to read me what it typed on screen. Right now the
| software expects you to check what it wrote and you have to
| necessarily keep looking at the screen.
|
| Call me lazy but I wish for a method where I could lay back,
| speak a sentence, wait for transcribing, listen to what was
| written and ahead I go.
| Snitch-Thursday wrote:
| when I was writing my last novel draft using nuance dragon 15,
| it had the 'read that' command which would read the last phrase
| I inputted, but that wasnt enough since I had a wireless
| headset that let me wander the house talking to the computer,
| but still needed to see the screen to know if it got the
| dictation correctly.
|
| I ended up fishing out a cheap $120 portable walmart windows
| laptop 2 in 1 and VNCing it into the desktop so i could see the
| screen, carrying that around with me worked.
| ethn wrote:
| This might be able to be a bit faster if you use a virtual QWERTY
| keyboard displayed with eye tracking acting as a cursor, with
| select being a consonant, preferably /d/. You can map certain
| phonemes for other shortcuts, including one for click/right-
| click/vim-commands.
|
| Articulating words takes too long, even subvocalization is, which
| has the advantage of only using non-voiced phonemes.
| joshwcomeau wrote:
| Appreciate all the discussion!
|
| I touch on this in the article, but I should say: I'm an edge-
| case when it comes to CTS. Most cases resolve spontaneously. Part
| of it is that my nerve dislocates when I bend my elbow, and this
| mobility causes a bunch of additional friction / inflammation.
| This happens naturally to about 13% of the population, and mine
| is particularly pronounced.
|
| All this to say, if you ever do start to get a burning or
| tingling in your elbows or wrists, or get numbness in the hands,
| it'll likely go away on its own, or with conservative treatment
| (a physical therapist is a great first step!).
|
| And, for those who also fall into my unlucky percentile,
| hopefully it helps to know that there are tools and a community
| that exist to let you keep working without needing to use your
| hands at all!
| bdickason wrote:
| Not sure if this helps but I had a similar issue with my joints
| (arm, shoulder, wrist) where I would scream in pain when lifting
| a small book or anything above 3 lbs.
|
| The issue was caused by my immune system acting up which would
| cause my tendons to become inflamed. After a year of
| misdiagnosis, Doctors found some heavy medication (methotrexate
| balanced with plaquenil) to help regulate it. One of the
| medicines side effects was listed as 'death' on the label.
|
| After a year of experimenting, I found that major diet changes
| (lots more hearty greens, way less sugar and carbs, no caffeine
| or alcohol), improved sleep, reduced stress (quit my stressful
| job) completely alleviated my symptoms. I would still have flare
| ups from time to time which I reduced via physical therapy /
| exercise (to strengthen muscles supporting my tendons).
|
| Just sharing as I had a somewhat similar condition and was
| surprised that the fix didn't have to be a pill.
| corobo wrote:
| I was misdiagnosed inflamed tendons and it turned out to be
| something else, hah
|
| HN Reader take this message: medicine can be a to and fro, go
| back if you're not convinced and get a second opinion if you
| think somethings still wrong :)
| slashblake wrote:
| Are you me?
| bdickason wrote:
| Maybe we're the HN equivalent of Fight Club?! Am I Brad Pitt
| or Edward Norton
| joubert wrote:
| Drastically reducing "added" sugars (which includes alcohol),
| whether end-user added or added as part of the industrial food
| "manufacturing" process does wonders for one's health.
|
| Reducing sugar intake reduces inflammation
| (https://www.health.harvard.edu/heart-health/the-sweet-
| danger...).
| snielson wrote:
| Can confirm. Cut out almost all sugar from my diet. I've seen
| dramatic improvements in my health with the most significant
| being the elimination of brain fog.
___________________________________________________________________
(page generated 2020-10-21 23:00 UTC)