[HN Gopher] Show HN: Science and math-flavored programming probl...
___________________________________________________________________
Show HN: Science and math-flavored programming problems for
beginners
Author : PolarizedPoutin
Score : 267 points
Date : 2021-04-17 11:08 UTC (11 hours ago)
(HTM) web link (projectlovelace.net)
(TXT) w3m dump (projectlovelace.net)
| robinj6 wrote:
| What do you use for the interpretation of so many languages? Did
| you just build a bunch of sandboxed vms?
| PolarizedPoutin wrote:
| Ah do we run all user-submitted code in Docker containers. The
| "engine" that runs the code is written in Python and we do
| different things for different languages.
|
| For running Javascript and Julia, it goes something like Python
| objects -> JSON -> read JSON in Javascript/Julia -> run code ->
| output JSON -> read user output from JSON in Python.
|
| For C, we can call C functions directly from Python with some
| code for dealing with different types.
|
| Not sure if this is the best approach (it's not super fast) but
| we've been learning as we go. We might be due for a refactor in
| case the next language we want to support doesn't fit into this
| pattern. I'm personally excited to learn Rust and maybe add
| support for it.
|
| PS: Our stack is on GitHub in case anyone is interested in
| having a look: https://github.com/project-lovelace
| Karliss wrote:
| From a quick glance it looked quite worrying, many red flags.
| I didn't look too carefully so some of this might be wrong or
| I missed where it's done.
|
| * Results from the untrusted part inside container are
| returned using pickle. Which can be used to achieve arbitrary
| code execution outside container.
|
| * no time limiting
|
| * no memory limiting
|
| * Untrusted code is run as root in the container which by
| default is same user as root outside container. From what I
| understand it isn't as bad as it was in earlier docker
| versions but still not great.
|
| * untrusted code is run in the same process as semitrusted
| run_lang code, which means that the untrusted code with
| little bit of effort can manipulate reported execution time
| and memory usage
|
| * for some languages correct_output is copied into the
| untrusted execution environment which means that solution
| could potentially just read the correct answers instead of
| calculating themselves
|
| * none of the default capabilities are dropped which is
| probably more than what solution needs
|
| * solution can freely create new files
| PolarizedPoutin wrote:
| Thank you so much for taking a look and letting us know!
|
| Definitely not surprised that there are many security
| vulnerabilities...
|
| I'm a grad student in computational science working on this
| as a hobby so building and securing websites is not
| something I know much about.
|
| But now that someone is actually using it, I will take
| security more seriously.
|
| Once I have some time I'll open a GitHub issue and compile
| a list of the security vulnerabilities you listed and will
| learn how to fix them.
|
| This should be a good learning opportunity!
| pvg wrote:
| _This should be a good learning opportunity!_
|
| That's a good attitude to have but people are probably
| going to take over your host[s], vandalize your site and
| run up your bills long before you get a chance to do all
| the necessary learning. I don't think your current
| approach is actually all that easy to secure.
| PolarizedPoutin wrote:
| I'm open to suggestions and advice if you have any.
|
| This website is a free open-source side project and not
| business critical.
| PolarizedPoutin wrote:
| Thanks for the suggestion. Thread is too deep to reply to
| your actual message, but I will look around to see how
| other "online judge" software run arbitrary code
| securely.
|
| There's probably some low hanging fruit in configuring
| Docker properly.
| pvg wrote:
| I don't have anything particularly concrete but I'd say
| find an open source implementation of something similar
| that has had a track record of running without too much
| incident and carefully copy its implementation, design
| and configuration.
| Karliss wrote:
| My suggestions are:
|
| Disable ability to make submissions until you have more
| solid plan
|
| Decide what is your goal. Do you want to make a judge
| system, do you want create tasks or do you want a
| platform with specific kind of tasks.
|
| Get in touch with people involved in ICPC an IOI contests
| in your country. Even if you are not interested in those
| kind of algorithm tasks, there will be some people who
| are familiar with similar existing systems and could
| point you in the right direction.
|
| Assuming your primary goal isn't to make a judge system
| itself, some other options are: * Evaluate the existing
| online judge systems. There are some open source ones
| like DOMjudge(https://www.domjudge.org/),
| CMS(https://cms-dev.github.io/index.html) and others .
| Consider if you reuse or extend them to suite your
| desired format. In the worst case maybe just the
| execution part can be reused. At least learn from their
| experience and mistakes creating and maintaining such
| systems.
|
| * Many programming languages now have online REPL
| environments. Some of them open source. This is one more
| source of projects that provide sandboxed execution.
|
| * If you have some budget, there are platforms that
| provide sandboxed exection as service oriented at your
| exact use case. Some examples are Sphere Engine
| https://sphere-engine.com/enterprise used by Sphere
| online judge and Kattis .
|
| There a lot more different platforms with different style
| of programming tasks than what you listed in your FAQ.
| Some of looking for problem setters. Maybe one of them
| fits your type of tasks more. Or it can be a one off
| contest with slightly unusual problem set. Or maybe it
| can be a separate category on their system and you can
| advertise this category on your website.
| 101008 wrote:
| How do you check for malicious code, or something simple as
| "while True"?
| PolarizedPoutin wrote:
| There should be a limit on how long the Docker container
| can run code for, but it might be unnecessarily long right
| now.
|
| I'm not a web developer by trade or anything so I'll have
| to learn how to secure the Docker container from malicious
| code. Hopefully Docker provides some amount of protection
| for now...
|
| For today it's going to be old school htop + F9 haha.
| jcpst wrote:
| As a curious polyglot, this is great.
| stonewareslord wrote:
| These problems look like a lot of fun! Unfortunately, I'm
| currently learning Rust and would prefer to use it to solve these
| problems.
|
| It would be nice to support uploading a binary or solution (like
| project Euler) or a CSV of test cases next to solutions. Maybe
| I'll try compiling rust to c and upload the c file
| a3n wrote:
| You could do the problems in rust, locally, and bask in the
| glow of accomplishment without communicating your results to
| the site.
|
| People self-study text books without sending in their solutions
| to the publisher.
| PolarizedPoutin wrote:
| I'm excited to learn Rust actually so we might support it soon!
|
| It's not trivial to add support new languages since the engine
| app that runs user-submitted code needs to be able to run
| arbitrary code in your chosen language and needs a way to
| communicate with Python (either directly or through JSON
| passing).
|
| So far we've just been adding support for languages we know and
| use.
|
| But indeed there are hundreds of languages out there that would
| be nice to support in some way. I'm gonna think about how we
| can add a new "submission mode" where you're given a couple of
| test cases that you run manually with any language then submit
| your solution.
| dghf wrote:
| In the Python templates, what's with the variable
| "initialisations" --- e.g. t = 0 in the light-speed one? I've
| seen similar stuff in my eldest's computing homework: is it an
| accepted Python idiom that I've missed somehow?
| PolarizedPoutin wrote:
| Yeah the other two replies explain why we used code stubs that
| would at least run but not pass.
|
| Don't think it's an idiom (at least not one that I've seen).
|
| It's definitely not the nicest code, especially for simpler
| problems that are really one-liners but we figured lots of
| people would just click "submit" on just the code stubs.
| Jtsummers wrote:
| It looks like they've made the minimum amount of code that will
| execute (but not pass the tests). By using a variable instead
| of just hardcoding 0 into the _return_ they give you a specific
| thing to assign to and as long as you leave that _return t_ bit
| alone at the end your code will work.
|
| Though, in that case, the solution is so trivial that _t_ is
| completely unnecessary.
| tremon wrote:
| I presume it is to prevent a NameError if you run the template
| without adding any code.
| yeowMeng wrote:
| I like the eclectic selection of problems. I wish I had seen
| something like this when I was growing up; a manageable set of
| problems with the hint of depth.
|
| It's easy to snipe a nerd, but nerd sniping a non-nerd.. That's
| gold.
|
| FWIW: - No margins on the side on iPhone in portrait. - Math
| formulas are clipped on the top in landscape.
| PolarizedPoutin wrote:
| Haha thanks for taking a look!
|
| Ah I'm no web dev so must have missed that margin issue. Was
| able to reproduce in tablet mode. Thanks for pointing it out!
| truth_ wrote:
| Wow. I really like this!
|
| Will begin solving all of them.
|
| And I would like this to exists for a long time and I want it to
| be around when I teach programming to my little cousins, nephews
| and nieces and my future generation.
|
| Add an option to Donate. Would love to do so.
| russfink wrote:
| What happens when you submit code?
| PolarizedPoutin wrote:
| The code gets uploaded to the server which sends it to an
| "engine" sitting in a Docker container. The engine generates
| test cases and send them with the code to another Docker
| container where the code is run. Then the output from your code
| is sent back to the engine which checks to see how many test
| cases you got correct before sending all the information back
| to your browser.
|
| You can check out the engine code on GitHub if you're
| interested in the details: https://github.com/project-
| lovelace/lovelace-engine
| amelius wrote:
| I implemented the Babylonian square root problem using a simple
| "x to the power of 0.5" and it accepted the answer!
| PolarizedPoutin wrote:
| Yeah haha, this is somewhat intentional!
|
| We thought it would be tedious to write tons of code to make
| sure users don't take the easy way out for each problem and
| language, so we figured we might as well allow it. We can't
| force users to solve the problem our way.
| dvfjsdhgfv wrote:
| Well done, thank you!
| cyberia23424 wrote:
| Awesome website! Although it often loads very slowly for me.
| PolarizedPoutin wrote:
| Thank you and sorry about the slow load time.
|
| We never expected this much traffic and everything is hosted on
| a tiny DigitalOcean server including the code runner haha. If
| you visit a bit later it should be much faster.
|
| I can think of some ways to speed things up which I will try.
| iamcreasy wrote:
| It looks cool. Thanks! I've started solving problem in Julia.
| Are there 27 problem in total?
|
| Also, I can not seem to submit solution to problem 'Compound
| interest'. I kept getting the following error, '...docker
| container is nonzero. Returning falcon HTTP 400.'. Sent you
| an email with the details.
| PolarizedPoutin wrote:
| Nice to see another Julia user! Yup right now we only have
| 27 problems but there will hopefully be many more in the
| future. Some people might even contribute some new
| problems.
| Vaslo wrote:
| This is great
| cbkeller wrote:
| This looks awesome! I might try to incorporate this as an extra
| resource in an intro (Julia) class in the future.
| PolarizedPoutin wrote:
| That sounds awesome! Definitely let us know if we can help.
| gfiorav wrote:
| Love this! Always been a fan of kata-type websites like [0], but
| they all become uninteresting after a while (new katas get
| bland). This submission has good fresh energy!
|
| [0] - codewars.com
| PolarizedPoutin wrote:
| Haha thanks! Didn't know about codewars.com, will look into it.
| Might be a cool way to learn Rust syntax.
| nxpnsv wrote:
| I got completely obsessed with the code golf part at
| codewars, at some point I just decided enough is enough and
| stopped playing with. A lot of fun though.
| nxpnsv wrote:
| It is neat. The inline math formulas are not displaying right
| (unless raw latex is right...). Compared to Euler I think perhaps
| the early problems are a little too easy...
| PolarizedPoutin wrote:
| Thanks for checking it out! Will look into the LaTeX rendering
| issue.
|
| Yeah maybe we went a little too easy on the earlier problems
| but there will be plenty of harder problems coming up
| hopefully!
|
| I actually think Project Euler gets hard pretty quickly but I'm
| no number theory wizard haha.
| domano wrote:
| Very nice curated set of problems with an extremly low entry
| barrier!
|
| Just would appreciate golang support, but i guess you will get
| feature requests for all languages anyways :)
| PolarizedPoutin wrote:
| Thanks for checking it out!
|
| I don't use Go so might take a while unless someone decides to
| swoop in and add support for it!
|
| Supporting new languages takes some effort since we need to be
| able to run arbitrary Go code and communicate with Python.
| Definitely possible but requires some familiarity with both.
|
| So far we've just been adding languages we know and use. Hoping
| to learn Rust soon so that might be the next language!
| Smaug123 wrote:
| One of the big reasons Project Euler is so brilliant is that
| every problem is formulated so that you can do it in any
| language. (I've introduced myself to about six languages this
| way, none of which you support yet.) What drove the decision to
| have an allowlist of languages? If you're concerned about people
| publishing the one true number that is the answer to problem 11,
| you could always generate random test cases and ask the user to
| supply the answers to those test cases.
| pontus wrote:
| The downside with Project Euler, though is that you need to run
| your code elsewhere. That's why something like leetcode is so
| nice (I think, at least): you can run the code right there in
| the browser. If you take this approach, you're forced to
| whitelist languages.
| Smaug123 wrote:
| Porque no los dos? Allowlist the languages you understand,
| also allow manual entry for the languages you don't? Seems
| like the obvious solution.
| PolarizedPoutin wrote:
| Right yeah it's a good question.
|
| I think the reason we did it is so that we could visualize user
| output/solutions. You can learn new things by looking at your
| solutions to various test cases!
|
| One example would be if you submit a solution (or just the code
| stub) to the Exponential growth problem, then your solution
| gets plotted and compared with the analytic solution and the
| correct solution:
| https://projectlovelace.net/problems/exponential-growth/
|
| Also, unlike Project Euler where every solution is a number, a
| lot of scientific problems have solutions that are multi-
| dimensional arrays or multiple objects. So in this case, copy
| pasting your output becomes quite messy.
|
| It's not trivial to add support new languages since the engine
| app that runs user-submitted code needs to be able to run
| arbitrary code in your chosen language and needs a way to
| communicate with Python (either directly or through JSON
| passing).
|
| So far we've just been adding support for languages we know and
| use.
|
| But indeed there are hundreds of languages out there that would
| be nice to support in some way. It might be neat to add a new
| submission mode where you're given a couple of test cases that
| you run manually then submit your solution manually.
| siltpotato wrote:
| So it's Project Euler but also CodeWars.
|
| You can even register for it!
| zakember wrote:
| Great site, lot of interesting problems.
|
| Any chance of adding Rust to the list of supported languages?
| Bostonian wrote:
| Could you add Fortran as a programming language? It is one of the
| main languages for "science and math flavored programming
| problems". Gfortran is a free compiler that is part of gcc.
| PolarizedPoutin wrote:
| That is true. It's not super easy to add new languages and I'm
| not familiar with Fortran (thankfully our group switched from
| Fortran to Julia recently!) but we already support C since it's
| easy to call C from Python. Perhaps Fortran support won't be
| super hard either.
| sundarurfriend wrote:
| Suggestion: allow entering the registered email ID in the place
| of username, when logging in. I was repeatedly trying to login
| and even reset the password, assuming the Username field was
| "Username or Email ID" like it is in many places now. This is
| certainly a bit of PEBKAC, but it would be a nice usability
| improvement to allow what's now become a common pattern and make
| either username or email ID work in that part of the login form.
| PolarizedPoutin wrote:
| Thanks for the suggestion! I'm all for quality of life
| improvements. Definitely wouldn't be hard to add I think.
| pontus wrote:
| Really nice! When I was in college we had all sorts of online
| systems that automatically graded assignments based on how close
| to the correct answer your supplied answer was. I think it'd be
| nice if in the future there was more of an overlap between
| science classes and programming classes (almost like a freshman /
| junior level scientific computation class) where instead of
| approaching problems from a pure theoretical perspective, we
| involved these types of computational approaches in parallel.
| Personally I have found that while I often know how to perform
| various theoretical computations, it's faster for me to just
| throw together a quick script to approximate the result (e.g.
| what's the expected value of the product of two gaussians?)
| PolarizedPoutin wrote:
| I agree 100%. It seems that almost any real-world science
| problem these days requires a computational solution/approach
| at some level. Taking a computational approach in parallel
| would be super useful.
| truth_ wrote:
| Just like the "Structure and Interpretation of Computer
| Programs", there exists a "Structure and Interpretation of
| Classical Mechanics".
|
| I think you might enjoy it. Visit- https://en.m.wikipedia.org
| /wiki/Structure_and_Interpretation...
| carldaddy wrote:
| I agree. I wish school would have merged things like space and
| programming. I would have been really into it.
| OkayPhysicist wrote:
| This was sort of how physics classes at my alma mater worked.
| Especially at the upper-division level, it was pretty much
| expected that you would be submitted a Mathematica notebook
| with all your homework/tests/labs
| hivacruz wrote:
| Really cool. Sure the exercises are simple but a little reminder
| of some science concepts it's always nice.
| PolarizedPoutin wrote:
| Yeah haha they're definitely simple if you have almost any
| education in math or science.
|
| I guess we started simple so that the barrier to entry is low,
| but we should have some tougher and more interesting problems
| soon!
| wiml wrote:
| It is already mentioned in Project Lovelace's About page, but
| people who like this and Project Euler may also like Rosalind,
| which is a programming problem site focused on bioinformatics and
| adjacent algorithms.
| sireat wrote:
| Love the little explanations on the left!
|
| I've used many of these same problems as small assignments when
| teaching programming to novices.
|
| However all my assignments look much uglier. And looks do matter.
| de6u99er wrote:
| That's cool, but I'd love if you'd add Java to the programming
| languages.
| PolarizedPoutin wrote:
| Java is definitely on the list of languages to support soon!
|
| I'm also thinking of how we can add a "manual submission mode"
| that lets you use any language that you run locally.
| cochne wrote:
| This is fantastic, I actually independently gave my CS101 (for
| engineers) students some of the same questions last semester
| (temperature, definite integrals, game of life). Can't wait to
| try some of these out!
___________________________________________________________________
(page generated 2021-04-17 23:00 UTC)