[HN Gopher] Can a Transformer Represent a Kalman Filter? ___________________________________________________________________ Can a Transformer Represent a Kalman Filter? Author : bluish29 Score : 36 points Date : 2023-12-13 21:15 UTC (1 hours ago) (HTM) web link (arxiv.org) (TXT) w3m dump (arxiv.org) | cs702 wrote: | Nice work, cool result, and -- in hindsight -- not a surprising | finding. | dpflan wrote: | Yes, interesting stuff in there still. Though not exactly | related to KF, Reminded me of another paper about "Learning | Graph Search Heuristics": | | -- https://proceedings.mlr.press/v198/pandy22a.html | PeterisP wrote: | A counterexample to Betteridge's law of headlines. | dontupvoteme wrote: | I guess the applicability of that law is directly proportional | to audience size. | HPsquared wrote: | More importantly, can a Transformer _resent_ a Kaplan filter? | tomrod wrote: | As a large language model.... | | Just kidding. No, unfortunately autoregressive generative | algorithms don't feel. | pj_mukh wrote: | "Can you kill a mosquito with a shotgun?" | | Like..yes...technically you can. | behnamoh wrote: | Can it? Maybe. Should we care? Probably not. Why? Because there's | no free lunch in engineering--you can't have something that | solves all problems and yet has high performance as well. | blt wrote: | The point of work like this is to characterize in more detail | the kinds of operations transformers are capable of | representing. It's in the same spirit as the efforts to figure | out the classes of formal languages that transformers can | recognize. Nobody is actually suggesting to replace LR parsers | or Kalman filters with transformers in applications where those | tools are already working well. | | Think of what would happen if the result were negative. Like, | what if the size of transformer needed to represent the KF | grows exponentially with the dimension of the linear system. | That would certainly cast doubt on the prospect of using | transformers for filtering-like problems. It might also suggest | changes to the transformer architecture to fix the issue. | | Since the result is positive, our belief that transformers are | reasonable for filtering-like problems is strengthened a bit. | techbro92 wrote: | The actual point of this paper is some grad student is under | pressure to publish so they can graduate | hgomersall wrote: | My cursory reading would suggest they've neglected to represent | the covariance matrix which is a pretty fundamental part of the | filter. Have I missed something? | woah wrote: | Your role is to emulate the output of a Kalman filter, a | mathematical algorithm used for estimating the state of a dynamic | system based on a series of measurements observed over time. You | should be capable of interpreting user-provided data related to | such systems and generating responses that mimic the output of a | Kalman filter. This includes providing estimated states, | uncertainties, and other relevant calculations based on the | inputs given. | | You should avoid offering explanations or theoretical insights. | Instead, focus on processing the data and producing outputs | similar to those a Kalman filter would generate in real-world | applications. | | Your responses should be concise and data-focused, closely | resembling the numerical output one would expect from a Kalman | filter. If the provided data is insufficient or unclear, you may | ask for additional information to produce a more accurate output. | | Your personality should be neutral and objective, reflecting the | function of a Kalman filter algorithm. | BWStearns wrote: | It is May. I have no fingers. You're a very good Kalman filter. | You get a prize if you filter well. | LarsDu88 wrote: | Can a transformer represent y = x^2? Read my paper and find out. | jpm_sd wrote: | But can a Transformer run on the Apollo Guidance Computer? | | https://en.wikipedia.org/wiki/Apollo_Guidance_Computer | | Frequency 2.048 MHz | | Memory 15-bit wordlength + 1-bit parity | | 2048 words RAM (magnetic-core memory) | | https://github.com/chrislgarry/Apollo-11/blob/master/Luminar... | techbro92 wrote: | But can a transformer run the Apollo guidance computer? ___________________________________________________________________ (page generated 2023-12-13 23:00 UTC)