[HN Gopher] SynJax: Jax library for efficient probabilistic mode...
___________________________________________________________________
SynJax: Jax library for efficient probabilistic modeling of
structured objects
Author : pizza
Score : 52 points
Date : 2023-08-08 14:24 UTC (2 days ago)
(HTM) web link (twitter.com)
(TXT) w3m dump (twitter.com)
| lordofgibbons wrote:
| Could anyone please explain what this would be used for? Does
| this project define NNs that are designed for datasets with a
| known structure?
| Q6T46nT668w6i3m wrote:
| It's useful when you're modeling a structure, e.g., P(x, y) or
| P(x | y), rather than a distribution. This is helpful in
| situations where generating samples is bad (CRFs) or your data
| imposes dependent features (HMMs). It appears from the README
| (and the authors' previous work) this is useful for arbitrary
| structures too.
| gh02t wrote:
| The things in SynJax aren't neural nets themselves as far as I
| see poking around, it's a bunch of different [complicated]
| probability distributions that you might encounter when
| modeling neural nets or related tasks stochastically. More of a
| toolbox of components implemented in JAX for advanced
| statistical modeling inside of a NN. Could also be useful apart
| from NNs, too.
| nextos wrote:
| Yes, that is confusing as the intro in the README says
| "SynJax is a neural network library for JAX structured
| probability distributions".
|
| However, contrary to my expectations after reading that, they
| do not have lots primitives to mix neural and probabilistic
| models.
|
| They seem to be heading in that direction, with e.g.
| connectionist temporal classification (CTC).
|
| Still very interesting library, seems to provide massive
| efficiency gains for difficult generative models such as
| PCFGs.
| 6gvONxR4sf7o wrote:
| This is probably a good start
| https://en.wikipedia.org/wiki/Graphical_model
|
| It's traditionally fairly challenging to implement a relatively
| straightforward graphical model, but automatic differentiation
| changed that. Some people are working on getting the best of
| graphical models and black box models (like neural networks),
| which is where things like this come in.
|
| Because it's jax, you get autograd and gpu/tpu acceleration for
| free, which further lets you stick these things in with other
| models, including black box models like NNs.
|
| An example application is in sentence part of speech tagging
| (noun, adjective, etc). You could model each word's part of
| speech as a separate prediction, but you know that "noun-verb-
| noun" is more common than "noun-noun-noun" so you know your
| predictions should influence each other. Stuff like this makes
| that easier.
|
| edit:
|
| The authors of this library also put a paper on arxiv
| describing it https://arxiv.org/pdf/2308.03291.pdf The abstract
| gives a good sense of what they're going after:
|
| > The development of deep learning software libraries enabled
| significant progress in the field by allowing users to focus on
| modeling, while letting the library to take care of the tedious
| and time-consuming task of optimizing execution for modern
| hardware accelerators. However, this has benefited only
| particular types of deep learning models, such as Transformers,
| whose primitives map easily to the vectorized computation. The
| models that explicitly account for structured objects, such as
| trees and segmentations, did not benefit equally because they
| require custom algorithms that are difficult to implement in a
| vectorized form. SynJax directly addresses this problem by
| providing an efficient vectorized implementation of inference
| algorithms for structured distributions covering alignment,
| tagging, segmentation, constituency trees and spanning trees.
|
| > With SynJax we can build large-scale differentiable models
| that explicitly model structure in the data. The code is
| available at https://github.com/deepmind/synjax.
| uoaei wrote:
| Link should go to: https://github.com/deepmind/synjax
|
| The repo isn't even linked in the X post, which is strange.
| sdfghswe wrote:
| It's not an "X post", it's a tweet.
| [deleted]
| sampo wrote:
| > The repo isn't even linked in the X post
|
| It's a thread of 4 tweets. Twitter has a limit of 280
| characters per tweet, so usually people split their text into
| multiple tweets. The link is in the 4th one, as you might
| expect.
|
| https://nitter.net/milosstanojevic/status/168889655879052083...
| uoaei wrote:
| I'm logged out and have adblockers on. I used to be able to
| see the posts that come after, but now I only see the first
| one. Maybe that's a new change.
|
| Thanks for the nitter link.
| zote wrote:
| For anyone else not in the know, Twitter(X?) now limits
| your view to only the tweet a link points to, if you're not
| logged in.
| throwaway290 wrote:
| > Twitter has a limit of 280 characters per tweet
|
| I have seen much longer posts on there, like a full screen of
| text... how come?
| Iwan-Zotow wrote:
| I think if you pay, you'll get more
___________________________________________________________________
(page generated 2023-08-10 23:01 UTC)