[HN Gopher] Computer vision basics in Excel, using just formulas
___________________________________________________________________
Computer vision basics in Excel, using just formulas
Author : alok-g
Score : 165 points
Date : 2020-02-18 16:49 UTC (6 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| lowdose wrote:
| Chart to data would be awesome. It takes a lot of time to adjust
| a png into an image possible to redistribute with custom
| branding. It's only surface detection so must be possible.
| yummypaint wrote:
| Very nice. Spreadsheets are also great for doing quick monte
| carlo simulations. Things like finding the solid angle of a
| cylinder from an arbitrary perspective quickly become
| algebraicaly intractible. Raytracing with gnumeric is comparably
| easy.
| animalnewbie wrote:
| I'm beginning to wonder who/what does/did more harm to
| technological progress - Microsoft or aversion to Microsoft?
|
| Sounds of the stuff they do is incredible, just not talked about.
| kevingadd wrote:
| Which of the two probably depends on the particular situation,
| as it often is in technology.
|
| Supposedly there was a time period where the Android team was
| considering .NET/C# instead of Java. In the short term their
| choice of Java was best for them but in the long run it led to
| things like the Oracle lawsuit and its threat of changing
| copyright law forever in a bad way - so it's interesting to
| consider what an alternate timeline would look like.
|
| IE6 was amazing at the time compared to its competitors, but
| then it quickly became a hindrance - a good example of how the
| situation can flip the moment the ecosystem changes.
| eigenvalue wrote:
| This is an amazing idea! It's also a testament to the extreme
| power and efficiency of the core Excel code that everything works
| so smoothly despite this being not at all what Excel was designed
| for. There is something about everything-- data and "code"--
| being so instantly and interactively available for inspection
| that makes everything seem simpler and easier to grasp.
| zweep wrote:
| 90% of the world's software developers don't realize that they
| are software developers and their language is Excel formulas.
| gugagore wrote:
| Could someone report how feasible it is to run this in
| LibreOffice Calc?
| ryanjshaw wrote:
| It's in the linked readme:
|
| "While the files open in LibreOffice (tested in version
| 6.4.0.3 (x64)), it is slow to the level of being unusable. We
| have not tested in Apache OpenOffice."
| whereistimbo wrote:
| In my opinion this might because of xlsx compatibility
| overhead. Using ods format might speed up a bit.
| dclusin wrote:
| Microsoft has had really sharp people working on
| spreadsheet performance for many years. I remember
| reading a blog post from I believe Joel Spolsky or
| someone talking about what excel is doing behind the
| scenes to achieve high performance and I was pretty
| impressed.
|
| One example that comes to mind was that spreadsheets are
| just memory mapped files and the layout of the file on
| disk is identical to the data structures in memory. This
| allows them eschew translation to a data interchange
| format. So they got performance at the cost of
| interoperability, which is probably what's hampering open
| office & friends.
| blattimwind wrote:
| > One example that comes to mind was that spreadsheets
| are just memory mapped files and the layout of the file
| on disk is identical to the data structures in memory.
| This allows them eschew translation to a data interchange
| format. So they got performance at the cost of
| interoperability, which is probably what's hampering open
| office & friends.
|
| That improves save/restore performance, but in and of
| itself doesn't do much about execution performance of
| macros.
| dclusin wrote:
| It would help program runtime as well because the macro
| could be operating on data not loaded in memory. Not
| having to marshal and un-marshal data can save lots of
| execution time, especially in the face of non-contiguous
| reads/writes. Databases store information using B-Tree's
| for example so that they can calculate the offset of the
| data and jump directly to it. It would probably take a
| lot of gymnastics to get this from an XML or JSON file
| interchange format.
|
| I'm sure they could probably come up with something that
| is both portable and performant but it's probably not a
| big priority at Microsoft.
| derefr wrote:
| IMHO there's no reason to memory-map the interchange
| format itself.
|
| I would predict/expect that _both_ LibreOffice and MS
| Office (with their modern XML-based formats) are actually
| mmap(2)ing some temp file and treating it as an "on-disk
| working-state heap", and then importing from interchange
| formats by allocating from that heap / exporting to
| interchange formats by chasing pointers that end up
| inside that heap. (This is, after all, what every RDBMS
| does for _its_ working state. It 's pretty optimal.)
|
| Even if you have a memory-mapped interchange format, I'd
| still expect them to have a _separate_ disk-backed
| working heap for all the stuff that doesn 't belong in
| the file but is nevertheless very large (e.g. cached
| intermediary computation results of spreadsheet cells);
| and, if they have it, they may as well just use it for
| most things by default. Thus, I would expect that even in
| old versions of MS Office, the in-memory data structures
| were actually an interchange format _of sorts_ --not the
| ones being updated with each keystroke, but rather ones
| that'd be memcpy(2)ed into on export. (This also prevents
| you from either having to add a page-table structure to
| your file, or else constantly "defragment" it as data
| structures change.)
| alok-g wrote:
| OP co-author here. It could be worth a shot to try (I'll
| do when time permits).
|
| However, I am doubting it would help given how often
| LibreOffice was freezing with it (without crashing). My
| suspicion is that the formula recalculation is triggering
| more often that it should, and I could not find any way
| to prevent it. Excel provides options for manual
| triggers; if Calc does too, that might just solve the
| problem.
|
| PS: I am not deeply familiar with LibreOffice. :-)
| zweep wrote:
| Also Microsoft makes tens of billions of dollars per year
| from Excel and even if only 1% of that goes back into
| performance, that's a lot of optimization.
| alok-g wrote:
| Indeed. I have found Excel to be surprisingly fast at running
| this.
|
| I can also draw parallels to literate programming in this: The
| code and visualization of what the code is doing are very
| tightly integrated.
| BrandiATMuhkuh wrote:
| This reminds me of a fast.ai video where they use Excel for a
| CNN.
| BrandiATMuhkuh wrote:
| Found the video: https://youtu.be/gbceqO8PpBg
| wiseleo wrote:
| Interesting how they published a paper on GitHub. I wonder who
| else will adopt this format.
| stared wrote:
| See http://www.deepexcel.net/ - and educational April Fool's Day
| from 2016.
|
| I used to show these spreadsheets to make it explicit that all
| operations are simple, as in addition, multiplication, max and
| ReLU.
| sandesh1712 wrote:
| insightful and commendable effort to explain a complex topic of
| computer vision and CNN with lucid simple hands on step wise
| example in excel
| dsalzman wrote:
| If you want to see the power of "basic" operations. Watch this
| video of Dan Ingalls, co-inventor of Smalltalk, demo his software
| to do OCR on Devanagari text in 1980! https://vimeo.com/4714623
| StreakyCobra wrote:
| Remind me of this video of Matt Parker (standupmaths):
| https://www.youtube.com/watch?v=UBX2QQHlQ_I
| alok-g wrote:
| OP co-author here. :-)
|
| Yes, someone told us about this video when we first showcased
| this work. This video and a few more such works that we have
| discovered since then are linked in Q&A #7 in the readme. :-)
| LegitShady wrote:
| The text mentions a talk you gave. Was that recorded
| anywhere?
| alok-g wrote:
| We don't have a good-quality recording of this that we
| could release. I have been looking forward to making one in
| the future. Thanks for the interest. :-)
| StreakyCobra wrote:
| Congrats for this work! I haven't read the page far enough
| before commenting it seems (: Anyway, I hope this offered 13
| minutes of fun to some HN readers
| godelmachine wrote:
| The idea of transposing the 2-dimensional structure of image to
| the 2 dimensionalities of MS Excel is very intuitive.
___________________________________________________________________
(page generated 2020-02-18 23:00 UTC)