[HN Gopher] Tensorflow on edge, or - Building a "smart" security...
___________________________________________________________________
Tensorflow on edge, or - Building a "smart" security camera with a
Raspberry Pi
Author : ajsharp
Score : 89 points
Date : 2020-06-21 17:18 UTC (5 hours ago)
(HTM) web link (chollinger.com)
(TXT) w3m dump (chollinger.com)
| szczepano wrote:
| esp32-cam microcontroller costs around $6 and you have face
| detection build in. It have bluetooth and wifi and most of it's
| drivers code if not everything is on github. Only problem is you
| need to program it using arduino or other microcontroller
| hardware.
| ed25519FUUU wrote:
| Face detection is nice but body/person detection is much more
| useful in these setups.
| szczepano wrote:
| You mean esp-who ? It uses mobilenetv2 so it's quite possible
| to train it to detect person instead of face. Didn't tried
| myself, just started playing with it.
| brutus1213 wrote:
| Nice writeup but the Raspberry Pi isn't running tensorflow. It is
| mentioned in the article that the author is sending images to an
| edge machine.
|
| The big question I had was about hardware video encoding/decoding
| ... doesn't really cover that. I've found sending single image
| frames over zeromq to be fairly limiting if you care about high
| frame rate/low latency processing.
|
| Key issue I have run into is while many chips support hardware
| video encoding/decoding, the APIs to interface with this aren't
| there or not in open source. Anyone who has ideas on this, I'd
| welcome your comment.
|
| As an aside, another option is to run Intel's Movidus USB stick
| (aka Neural compute stick) and then you get a smart camera on the
| raspberry pi itself. That raises other issues though.
| barake wrote:
| Curious what you found limiting about zeromq? Just not enough
| throughput for high FPS?
|
| So far I've found it to be the sanest multicast solution since
| clients pull.
| brutus1213 wrote:
| Issue isn't ZeroMQ. The simple/inefficient way to do it is to
| capture frames one at a time, and send them via ZeroMQ. Video
| is pretty bandwidth intensive .. the only reason things like
| YouTube work as smooth as they do is that they use codecs
| such H264/265 (which are proprietary unfortuantely) and
| stream compress frames over the network. Now doing the codec
| in software burns a lot of CPU as this is very math intensive
| .. most processors support hardware video codecs for this
| purpose. There are just no open source tools/libraries that
| make this good/simple enough that I have found.
| ww520 wrote:
| Aren't IP cameras capable of H264 already encoded the video
| before output it to the network? H264 video stream has very
| good compression ratio and shouldn't consume too much
| bandwidth.
| thebruce87m wrote:
| This is where GStreamer normally steps in. A lot of hardware
| manufacturers provide a gstreamer plugin for their module. I've
| had experience with NVIDIA and atmel SoCs and that seemed to be
| the default path.
|
| Good luck with the gstreamer pipeline learning curve however!
| NikolaeVarius wrote:
| Can confirm not fun
| snowzach wrote:
| Shameless plug, check out DOODS:
| https://github.com/snowzach/doods It's a simple REST/gRPC API
| for doing object detection with Tensorflow or Tensorflow Lite.
| It will run on a Raspberry Pi. It actually did support the
| EdgeTPU hardware accelerator to make the Pi pretty quick for
| certain models. They broke something so I need to fix EdgeTPU
| support but it's still usable on the Pi withe the mobilenet
| models or inception if you're not in a hurry.
| milofeynman wrote:
| For people running Blue Iris you can do something similar to this
| with blue iris, deepstack[0], and an exe[1] someone wrote that
| sends the images to deepstack.
|
| Video guide: https://youtu.be/fwoonl5JKgo (Links in the comments
| of video as well)
|
| [0] https://github.com/gentlepumpkin/bi-aidetection
|
| [1] https://ipcamtalk.com/threads/tool-tutorial-free-ai-
| person-d... https://github.com/gentlepumpkin/bi-aidetection
| milofeynman wrote:
| I miss linked deepstack, sorry: https://deepstack.cc/
| fareesh wrote:
| I am trying to achieve something similar but at a higher scale.
|
| I have about 48 different cameras where I want to count people
| and get their approximate location in the frame.
|
| I want to run an object detection model on all of those video
| streams simultaneously.
|
| My AWS instance maxes out after 7 simultaneous streams so I
| figured I don't really need real-time monitoring. One frame every
| couple of seconds, even every minute could potentially suffice,
| since I am dealing with larger time-frames. Since I don't want to
| run too many instances at the same time, what are some viable
| strategies to achieve this?
|
| My plan is to have 5-6 instances of the ML model loaded up and
| waiting to accept a frame. When one of them is ready, it will
| instruct one of the RTSP streams to send it a frame, which it
| will process and store / send the result to an application
| server. I feel like I may not even be able to consume so many
| RTSP streams at once (I've never tried so I don't know), so I may
| have to have some other method of priming the handshake etc.
| before the model asks for a frame to process.
|
| Is there a better / non-hacky way of achieving this (i.e.
| managing the workload on a single GPU instance) ?
|
| I don't have any control of the camera hardware at all.
| theblackcat2004 wrote:
| Since you already using openCV, you can write a neat motion
| detection and only start sending frames for detection when
| motions are detected
| simlevesque wrote:
| Good thinking.
| thebruce87m wrote:
| Is it really edge computing if the pi isn't running tensorflow? I
| know the definition is kind of woolly.
|
| I wonder what the performance would be on a $100 jetson nano.
| dnautics wrote:
| it's not going to the cloud, so I'd say that counts.
| [deleted]
| brutus1213 wrote:
| Jetson nano can run Cuda code. It is pretty decent.
|
| The article does employ a reasonable definition of edge
| computing IMO (scientist who works in this area). The rpi is
| the client, and the processing happens on a beefy edge node.
| But yeah .. there is not one clear, accepted definition here.
| hn_check wrote:
| The Jetson Nano is fantastic, and I use Tensorflow on it with
| two home security cameras. It is a wonderful device, and
| deserves far more attention than it gets.
|
| I'm going to upgrade in the next week to a Jetson Xavier NX.
| Not because I need to, but because I like playing around and
| it's a silly powerful device.
|
| I also run a NextDNS CLI client on it, various automation
| stuff, etc.
| dheera wrote:
| Can we all _please_ stop using the term "edge" computing? It's
| nothing but a hype term and in reality it's really what we
| already had for the decades before the internet.
| noja wrote:
| Off topic almost: does anyone know of any (long life) battery
| powered wifi cameras (with IR) for a project like this? Off the
| shelf, with a battery life of months and nice looking (like Arlo)
| but not cloud?
| xchip wrote:
| the Raspberry Pi isn't running tensorflow
| canada_dry wrote:
| I'm hoping advances like YoloV5 [i] will allow a rpi4 to more
| ably do this without piping the video to another processor.
|
| [i] https://github.com/ultralytics/yolov5
| simlevesque wrote:
| Tensorflow on the pi itself is hard but I get great results for a
| similar system with just a rpi4 and opencv.
| alkonaut wrote:
| Would object detection like this work out of the box for deer
| like he demonstrates for humans? I need this for deer.
| joshu wrote:
| I've just started down this path using a plain RTSP-serving
| camera and a low end box using the Coral EdgeTPU to process the
| frames. It looks like there are a variety of solutions available.
| https://github.com/blakeblackshear/frigate
| https://docs.ambianic.ai/users/configure/ etc
| staycoolboy wrote:
| I've done this with a Jetson Xavier, 4 CCTV cameras and a PoE
| hub. You really want to use DeepStream and C/C++ for inference,
| not Python and TensorFlow.
|
| I'm streaming ~20 fps (17 to 30) 720P directly from my home IP4
| address, and when a person is in-frame long enough and caught by
| the tracker, a stream goes to an AWS endpoint for storage.
|
| I've experimented with both SSDMobileNet and Yolo3, which are
| both pretty error prone but they do a much better job filtering
| out moving tree limbs and passing clouds, unlike Arlo.
|
| You need way more processing power than an RPi to do this at
| 30fps, and C/C++, not Python. (There are literally dozens of
| projects for the RPi and TFlow online but they all get like 0.1
| fps or less by using Flask and browser reload of a PNG... great
| for POC but not for real video)
|
| I wrote very little of the code, honestly: only the capture pipe
| required a new C element. I started with NVidia DeepStream which
| is phenomenally well-written, and their built-in accelerated RTSP
| element, and added a custom GStreamer element that outputs a
| downsampled MPEG capture to the cloud when the upstream detector
| tracks an object. NVidia also wrote the tracker, you just need to
| provide an object detector like SSDMobileNet or YOLO. NVidia gets
| it.
|
| The main 4 camera-pipe mux splits into the AI engine and into a
| tee to the RTSP server on one side and my capture element on the
| other side.
|
| It was amazingly simple, and If I turn the CCD cameras down to
| 720P with h265 and a low bitrate, I don't need to turn on the
| noisy Xavier fan. The onboard Arm core does the detected
| downsampling (one camera only, a limitation right now) and pushes
| the video with a rest endpoint on a node server in the AWS cloud.
|
| I'm very pleased with it, I haven't tested scaling but if I
| turned off the GPU governors I could easily go to 8 cameras. I
| went with PoE because WiFi can't handle the demand.
| thebruce87m wrote:
| I'd love to read a write-up of this.
| exhilaration wrote:
| Do you have any links you could share to build something like
| this?
| reichardt wrote:
| TensorFlow Lite with SSDLite-MobileNet gets you around 4 fps on
| a Raspberry Pi 4 (23 fps with a Coral USB Accelerator):
| https://github.com/EdjeElectronics/TensorFlow-Lite-Object-De...
___________________________________________________________________
(page generated 2020-06-21 23:00 UTC)