[HN Gopher] Using Lidar to map tree shadows
___________________________________________________________________
Using Lidar to map tree shadows
Author : tppiotrowski
Score : 115 points
Date : 2023-07-09 19:28 UTC (3 hours ago)
(HTM) web link (tedpiotrowski.svbtle.com)
(TXT) w3m dump (tedpiotrowski.svbtle.com)
| Soupy wrote:
| This is ridiculously cool, demo is slick and fast! I work pretty
| extensively with tiling pipelines and GeoTiffs right now as I'm
| building out a mass repository and platform for historical map
| and aerial analysis (https://pastmaps.com - still very early so
| don't judge it too hard please)
|
| As part of this work, I similarly hit problems with using the raw
| GeoTiff files as my source and found that I was able to build
| some custom tiling hooks into MapLibre coupled with http range
| queries on the static files hosted in S3 to bypass the need for
| tiling. It does push the compute to the clients but I've found
| it's actually pretty fast even on older mobile devices.
|
| If this MapLibre GeoTiff source support is at all of interest,
| I'm happy to share some basic code or even open source some of
| that work. Here I was thinking I was the only weird dude on the
| internet messing with this stuff :D
| avibryant wrote:
| See also https://geoblaze-gsoc.vercel.app and the underlying
| libraries it uses, which also do range queries on GeoTIFFs.
|
| I'm super interested in this space, including in helping
| financially support some projects. I already emailed Ted about
| this, but would be happy to chat to anyone doing this stuff. My
| email's in my profile.
|
| (Luckily this is niche enough I'm not worried about my inbox
| blowing up....)
| tppiotrowski wrote:
| Ah. I hadn't even thought that I could do byte offsets on the
| fly. In general I would err on the side of over-sharing because
| until ChatGPT came along I struggled to understand what tools
| and options were to generate tiles as there wasn't much data I
| could find online.
|
| One thing specific in my case: the LiDAR GeoTiffs are in
| imperial feet with a 32 bit floating point precision. If you
| take the elevation range from sea level to Everest in meters
| (8848) and pack it into an int16, you can get .2 meter
| precision. That's plenty for ShadeMap so converting from
| float32 to int16 should save half the cloud storage space in
| theory and more when taking PNG compression into account.
| Soupy wrote:
| This code is _rough_ , and that's an understatement, but
| here's a basic gist showing how I'm doing what I'm doing in
| case it helps you in any way: https://gist.github.com/craigsc
| /fdb867f8971ff5b4ae42de4e0d7c...
|
| I'm similarly using R2 as my static hosting backend and it's
| been pretty fast and seamless
|
| note that i'm using the geotiff.js and fast-png libraries for
| the heavy-lifting behind the range-queries and the client-
| side png encoding. why reinvent the wheel, right?
| jamessb wrote:
| > coupled with http range queries on the static files hosted in
| S3 to bypass the need for tiling
|
| Did you look into Cloud Optimized GeoTIFF format?
|
| https://www.cogeo.org/
|
| It is supported by OpenLayers:
| https://openlayers.org/en/latest/examples/cog.html
|
| (I don't think that Maplibre or Leaflet have built-in support
| for it)
| Soupy wrote:
| I did actually! For my particular source dataset it was far
| easier just to lean on the plain old GeoTiff format since
| that's what my source data was already formatted in and my
| testing of the client-side on-the-fly tiling using range
| queries and fast-png for encoding resulted in close to par
| performance with 0 increase in hosting costs and 0 headache
| in building custom pipelines to encode the cloud-optimized
| versions. Basically, I'm lazy, I'm sure it's the "correct"
| answer for other use-cases
|
| I also have been digging pretty extensively into protomaps
| though for some newer non-GeoTiff datasets I'm in the process
| of pulling in, in my opinion it's the future for this space -
| https://protomaps.com/
| veec_cas_tant wrote:
| Wow, this is very neat. Awesome project
| punnerud wrote:
| Something that could be converted into SQLite database and served
| as static file at a fraction of the cost?
| https://news.ycombinator.com/item?id=27016630
|
| Range request in HTTP is doing the magic, but that part is
| already "solved"
| Soupy wrote:
| that's essentially the route that Mapbox went down and they've
| even invented an entire mbtiles file format that essentially is
| just a sqlite db for doing these kinds of queries on the server
|
| it's the "status quo" approach today in the industry, but it
| has some downsides still especially for smaller builders (ie
| me):
|
| 1. i'd have to run a separate tile server to take in the tile
| requests and convert them to "sql" requests (or mbtile
| requests) under the hood, and I'm just not a fan of more moving
| parts 2. I'd have to take my 10+ TB (and growing) of geotiffs
| and process them all into mbtiles which is a huge compute and
| walltime cost 3. the resulting mbtiles end up being similar in
| size at best, but at worst far larger than the original
| geotiffs so it balloons up the hosting and egress costs in
| exchange for faster requests. this is a great compression
| optimization breakdown for geotiffs that dives into this if
| you're interested -
| https://blog.cleverelephant.ca/2015/02/geotiff-compression-f...
|
| anyways, I'm sure ted has his own thoughts as well but this is
| at least what I've taken away from this space after diving into
| it with fresh eyes in the past few months
| verelo wrote:
| I love Shademaps. I wish my product that has been using it was
| more of a hit, but I'll say this, Ted and Shademaps are cool.
| Adding trees is super practical, in many of the areas I leverage
| this tool we're in Urban centres but in scenarios where i'm not,
| the tree data is almost always more relevant than the buildings
| or elevation (Ontario is pretty flat and thats 99% of my users)
|
| Keep up the great work!
| tppiotrowski wrote:
| Thanks Andrew. Still struggling a bit with the
| business/marketing side of things but the idea appears popular
| and the work itself is fun and fulfilling.
|
| Thanks again for your support.
___________________________________________________________________
(page generated 2023-07-09 23:00 UTC)