[HN Gopher] Datasette Lite: a server-side Python web application...
___________________________________________________________________
Datasette Lite: a server-side Python web application running in a
browser
Author : simonw
Score : 143 points
Date : 2022-05-04 15:22 UTC (7 hours ago)
(HTM) web link (simonwillison.net)
(TXT) w3m dump (simonwillison.net)
| tomatowurst wrote:
| absolutely insane. seems like its installing pip packages and
| running localhost in the browser? I don't 100% grasp what is
| happening underneath the hood. But usually you would launch the
| local server, browse over to http://localhost to view it in your
| browser. This is running the web server inside Web Assembly
| inside your browser? Is it then possible to expose that local web
| server inside web assembly to the internet? Crazy stuff! looks
| like its possible to speed up the initial load time by using web
| workers, very much eager to see that in action.
| Loading... distutils already loaded from default channel
| Loading micropip, pyparsing, packaging Loaded micropip,
| pyparsing, packaging Loading ssl, openssl Loaded
| ssl, openssl distutils already loaded from default
| channel pyparsing already loaded from default channel
| Loading setuptools Loaded setuptools
| simonw wrote:
| That's pretty much what it's doing!
|
| It runs everything in a web worker at the moment, not sure if
| there are optimizations I could make by running more of them in
| parallel?
|
| It doesn't exactly run a localhost server. It uses a mechanism
| from Datasette's internals that lets you simulate an HTTP
| request through the Datasette application and get back the
| result. Then it sends that result back as a message from the
| worker to the parent page.
|
| It's using this API here for that:
| https://docs.datasette.io/en/stable/internals.html#datasette...
| [deleted]
| detaro wrote:
| Please re-read the site guidelines and don't do this:
| https://twitter.com/simonw/status/1521878251848683520
| simonw wrote:
| For anyone wondering, my tweet here (which I deleted just
| before I saw this comment) was asking for help with votes on
| Hacker News.
| pstuart wrote:
| SQLite is so beloved that it's a natural for upvotes here ;-)
| tomatowurst wrote:
| I don't see anything wrong with this. We regularly see YC
| companies getting upvoted.
| pseudosavant wrote:
| Really cool idea and implementation! It is really making me think
| about if this can be done in a service worker so that you could
| expose what would look like a standard (REST?) API?
|
| It seems like a client-side non-JS server backend would be
| incredibly useful. Just drop the service worker in and point it
| at your data source.
| simonw wrote:
| I tried building this with a Service Worker first and it didn't
| work, because Pyodide needs XMLHttpRequest.
|
| I opened an issue about that here:
| https://github.com/pyodide/pyodide/issues/2432
| rmnclmnt wrote:
| Pyodide is unlocking a new era for the Python data ecosystem! So
| awesome Datasette is jumping on the train!
| nineteen999 wrote:
| I wondered for a moment why it was named after the Commodore
| Datasette:
|
| https://en.wikipedia.org/wiki/Commodore_Datasette
|
| I hadn't heared of datasette.io before. Looks like an interesting
| product.
| thadk wrote:
| How close is Python SQLite and Datasette Lite to accessing a
| hosted SQL database using HTTP range requests as can be done in
| sql.js like https://github.com/phiresky/sql.js-httpvfs?
|
| I put together a Pyodide-based web app where users need a few
| indexed queries from a 600mb SQLite database but it isn't very
| practical for them to download the whole thing into the browser.
| I would datasettify it and access it by API, if I wasn't relying
| on a pyodide library that needs direct SQLite access.
|
| https://observablehq.com/@thadk/life
| simonw wrote:
| I have an open issue for that here:
| https://github.com/simonw/datasette-lite/issues/28
|
| My initial hunch is that this will be really difficult -
| probably require a fork of something like
| https://github.com/coleifer/pysqlite3 then compiled for
| WebAssembly.
|
| I'm confident it's feasible, but I don't have the skills to
| figure it out myself.
| db65edfc7996 wrote:
| This was also my immediate thought. Being able to statically
| deploy larger databases without a server process would be an
| absolute dream. Approaching dangerously high levels of dark
| magic.
| gavinray wrote:
| It really blows my mind that you could (mostly) just up and shove
| the Datasette code into a browser. What a time to be alive.
| simonw wrote:
| Yeah, ditto!
| [deleted]
| almost wrote:
| That's really cool
| andrewmcwatters wrote:
| The datasette tools are really great. I use them for pulling
| GitHub statistics, such as the distribution of GitHub followers,
| from their REST API into an SQLite database with sqlite-
| utils(1).[1]
|
| Simon is in the 99th percentile (PR=0.998, n>25,000+) of GitHub
| users by followers.
|
| Thanks for such great tools, Simon!
|
| [1]: https://github.com/andrewmcwattersandco/github-statistics
| simonw wrote:
| Wow, I didn't know SQLite databases could compress that well:
|
| github-users.tzst - 96MB
|
| Uncompressed:
|
| github-users.db - 992MB
| rodcodes wrote:
| eatonphil wrote:
| My biggest issue with Pyodide (which is of course an awesome
| project/build on its own) is the long wait times. I haven't
| figured out a way around a ~5 second load time _where the entire
| UI hangs_ every single time you load the page.
|
| My app (similar to Simon's, a lite mode of a data IDE):
| https://app.datastation.multiprocess.io.
|
| My code:
| https://github.com/multiprocessio/datastation/blob/main/shar....
| StreakyCobra wrote:
| If you use it in a web worker the UI does not hang. It requires
| a bit more setup though:
|
| https://pyodide.org/en/stable/usage/webworker.html
|
| Edit: typo
| kencausey wrote:
| Related: https://news.ycombinator.com/item?id=31259027
| gavinray wrote:
| Damn, I had an idea to build a very similar thing since last
| week. There's nothing new under the sun.
|
| Really impressive app though, great job.
| simonw wrote:
| Running in a Web Worker can help a lot here - you can at least
| avoid locking the main browser UI thread and show your own
| loading indicator.
| simonw wrote:
| You can try it out here: https://simonw.github.io/datasette-lite/
|
| Or if you have a SQLite database hosted online somewhere with
| open CORS headers you can feed it the URL using ?url=... - like
| this:
|
| https://simonw.github.io/datasette-lite/?url=https%3A%2F%2Fc...
| riyadparvez wrote:
| Hi Simon, thank you for the great work. Do you have any plan to
| support DuckDB in future?
| simonw wrote:
| I am seriously considering adding a plugin hook to support
| alternative database backends - PostgreSQL and DuckDB are the
| two I'm most interested in for that.
| tomthe wrote:
| Hi, I work in an academic setting and I found datasette always
| interesting to publish datasets, but it is hard to host a
| python web application on institutional servers (would have to
| convince several people who would have to maintain this).
| Hosting static files is very easy - time to try datassete!
| simonw wrote:
| Awesome! I'm really keen to see Datasette used more in
| academia, I hadn't thought about how much easier it would be
| to deploy static files in that context.
___________________________________________________________________
(page generated 2022-05-04 23:00 UTC)