SCREWY SCRIPTS It's funny how I have ideas for posts that I note down but never get around to writing, then someone else writes about the exact same topic. I was going to write a retrospective post around the 19th of Oct., which is when I started writing this Phlog (though it wasn't online yet at that stage), but it was going to be a long post and I haven't wanted to spend that sort of time on it. But the other day I saw that Undo has written a similar sort of thing: gopher://sdf.org:70/0/users/undo/j058 But I've still been spending way too much time at the computer lately so I won't start writing that now. Actually he talks about not finishing projects. Of course in that regard I cheat a lot by describing in the ideas section projects that I don't actually plan to start in the first place, but one that has caught me lately is that internet client thing. I haven't talked about it much since describing the idea for the same reason, but I have set it up. Not actually on the Atomic Pi because that only arrived a couple of weeks ago so I set it up on a laptop instead. Except being left on all day it only survived for two days before overheating and being reduced to "system board error" BIOS beep codes, and my only other x86_64 laptop was the one that I had just finished setting up to replace my old 19 year old Thinkpad. So I kept using the old laptop for a bit longer. Then the Atomic Pi arrived, but it only boots UEFI, and I'd set up the system on the Micro SD card the old legacy BIOS way (which is all I've ever known to be honest). So yesterday I finally set up a copy of the card with a manual legacy/UEFI dual boot (or whatever you call that) installation, and I'll try that out later. No I won't because I've started babbling and before long I'll have been typing all afternoon. So shut up and get back on topic! Alright grumpy, I guess I'm actually just here to share some shell scripts. In spite of all my talk about being sick of messing about at computers (actually mainly as something to do in breaks from adding up expenses my the tax return, which I did finally get submitted), when I saw the Unix 1-liner Five Fridays Funfair I thought that was a great incentive to work on a bare-bones version of a YouTube browser that I've been intending to write ever since YouTube went Javascript-only a few months ago (I'm surprised nobody else on Gopher has been complaining about that - I bet they would if they'd been trying to browse it from a 19 year old Thinkpad with a 1GHz PIII CPU though!). Plus I missed out on the last Gopher event that I was going to participate in because that laptop is about the only portable computer that I have with a working battery and hefting it around the great outdoors for ROOPHLOC didn't seem too appealing. I did think about trying to use one of the Nintendo DS consoles that I've collected, but that was going to be quite tricky. Actually as of this week I have a box full of quite recent laptops including some little ones that would be quite well suited, but that's because out of desperation I'm going to try getting into the used laptop/parts business - which basically means becoming the nerdy equivalent of a used car salesman. Well I might borrow one later to, err, "test the keyboard", and make a late ROOPHLOC entry. gopher://katolaz.net/1/U1FFF_2020/entries.gph This was meant to be about scripts or something wasn't it? Oh yeah, so I tried to make a one-liner YouTube browser that processes the metadata output from youtube-dl, but it soon became obvious that it was going to be way over the 128 byte length limit. But I kept working on it anyway, adding features, and then I realised a way to cheat. There's no limit on the length of the example, so just define key parts of the script ahead of the one-liner that actually uses them all. It was still a little tricky getting that "main" function down to size, but I managed it with a byte to spare, and thought it was a good joke. I'm not sure whether "The Unix Wizardry Trustworthy Fellowship" really got the joke though, or didn't approve, because they just left out the example entirely. So it's pretty much useless as presented there. Hence I'll put the full version of the multi-line one-liner here, and a more practical version that runs as a script in the scripts section (later): ################################################################## Youtube Browser in ELinks ytb (){ [ "${1##*[!0-9]*}" ] && $YTBD --playlist-end $1 "$2"|$YTBE|$YTBS || $YTBD "$1"|$YTBE|$YTBS;elinks $YTBF;rm -f $YTBF; } Usage: ytb [MAX RESULTS] URL Example: $ YTBD='youtube-dl -e -g --get-thumbnail --get-description --get-duration' $ YTBE='sed -e s/_webp// -e s/\.webp/.jpg/ -e s/.jpg\?.*$/.jpg/' $ YTBS='csplit -z -f /tmp/yttmp -b %03d.txt - /^[0-9:]*[0-5][0-9]$/1 {*}' $ YTBF='/tmp/yttmp*.txt' $ ytb (){ [ "${1##*[!0-9]*}" ] && $YTBD --playlist-end $1 "$2"|$YTBE|$YTBS || $YTBD "$1"|$YTBE|$YTBS;elinks $YTBF;rm -f $YTBF; } $ ytb 5 https://www.youtube.com/user/FILMAUSTRALIA Comment: In ELinks: "<" ">" to change tabs/results, "W" to wrap text. Enable option "document.plain.display_links" in ELinks config. for download/viewable video/image URLs. "less" could be used instead of ELinks, with navigation between results using the ":n" and ":p" commands. Some X terminal programs can detect URLs by themselves. ################################################################## It works with individual video links, playlists, user accounts, and worked with youtube-dl's search function except that seemed to break while I was actually working on this script (you used to be able to do eg. "ytb ytsearch5:tildeverse" for five videos about the tildeverse (maybe)), I just search DuckDuckGo with "site:youtube.com" instead, or video.google.com does still work without Javascript (though you can't disable the link redirects like you can with DuckDuckGo). Add the "-f" option to the youtube-dl command (YTBD) to specify the video format/quality you want so it only gives you one video download link. The numbers printed to the terminal are the lengths of the description files, printed by Csplit, they're handy as a progress indicator. I also submitted some other simple one-liners that I've mentioned before. And while we're here, of course this Internet Client thing has involved all sorts of minor scripting. The latest one was auto-login with telnet. First off: yes I want to use telnet - the whole point of this thing is to avoid needing software that becomes obsolete and SSH is an example of that which I've battled far too much with already. Also I do want to use passwords even though it's unencrypted because the most likely point of attack is from the internet, so at my router (running OpenWRT), and I think a hacker is more likely to try brute forcing usernames over telnet if they get in than sniff the network traffic for the password when it's sent. It's not much protection, against an attack that's unlikely anyway, I know, but it makes me happy. There are examples on the internet of using Expect for auto-login with Telnet, so I did this at first and it worked fine. But I don't have Expect installed everywhere, and it needs TCL, which needs a list of other dependencies, and these are old systems without much HDD space to waste. I tried using Chat, but I didn't get very far with that. Then I found a suggestion on a forum that, over many hours of messing with stty and other things, eventually evolved into this: ################################################################## #!/bin/sh #Telnet auto-login USER='[username]' #Note: can't find a way to make letters "C" or "p" work in password. PASSWORD='[password]' HOST='[hostname / IP]' localtty="`stty -g`" stty raw -echo intr ^[ start '' { \ sleep 0.5; \ echo "$PASSWORD"; \ sleep 0.2; \ echo "stty \"$localtty\" &> /dev/null"; \ cat; \ } | telnet -l "$USER" "$HOST" 2>&1 | cat ################################################################## The C and p thing has just got me stumped. It always reads C as "interrupt" and p as "start" and I can't seem to change that no matter what I do with stty. In the end I meekly changed the password to avoid using those characters. Change the sleep commands to "sleep 1" if your sleep only understands integers. It does print something like: '$ stty "2506:5:bf:8a3b:3:1c:7f:15:4:0:1:0:11:13:1a:0:12:f:17:16:0:0:2f:0:0: 0:0:0:0:0:0:0:0:0:0:0" &> /dev/null' on the terminal after log-in, but I don't think it's worth adding extra complexity to hide that. You can automatically run other commands the same way. Here's the Expect version that doesn't take offence at any password character that you use: ################################################################## #!/usr/local/bin/expect -f set timeout 20 set host "[hostname / IP]" set user "[username]" set password "[password]" spawn telnet -l $user $host expect "Password: " send "$password\r" interact exit ################################################################## I don't use Telnet for launching graphical programs (the Xcellent part of the idea), but Rexec which simply includes a "-p [password]" argument. Rexec (from GNU inetutils - which I'm getting quite familiar with now) works very well for a program that nobody seems to want to make distro packages for. The main problem I've found is that I originally tested this with Dillo on my Pi Zero (which some may remember I couldn't run Firefox on), assuming that would be equivalent to Firefox as far a X goes. Nope, Firefox sends a rendered image of the whole page view, which is basically like uncompressed video when scrolling the page. It's mostly usuable with 10/100 Ethernet (it probably helps that my screen resolutions don't exceed 1024x768), but a bit too painful over my 802.11g WiFi, so I've got Ethernet cables strewn about now. There's more to say about that, but I'll leave it until I do a real follow-up post about the Internet Client. It'll probably be titled something like "Me and my Xcellent Ideas". - The Free Thinker Oh almost forgot, here's everything I found on the web relating to YouTube browsing (I looked pretty hard for an existing solution that I liked, but I'm picky): * yt - Command-line browser written in Python https://www.hecticgeek.com/yt-youtube-browser-ubuntu-linux/ * youtube-viewer - Perl script, Command line or GTK2/3, requires API key from Google https://github.com/trizen/youtube-viewer http://www.webupd8.org/2015/02/youtube-viewer-complete-youtube-client.html * straw-viewer - Perl script, Command line or GTK3, no API key required https://github.com/trizen/straw-viewer * SMtube - Qt GUI, latest version probably uses youtube-dl back-end http://www.smtube.org/ https://packages.debian.org/sid/smtube -Some sort of interface here: http://www.tonvid.com/index.php -Is it just a web-based proxy with a Qt browser reading from localhost? Or from that website? -Just a browser for the website, and the site itself is closed-source: https://sourceforge.net/p/smplayer/bugs/695/ The TONVID site might not be very reliable [indeed it hasn't been], but seems to work OK as a stop-gap. * Dillo-ydl - youtube-dl interface for Dillo written in Python https://git.scuttlebot.io/%25oEzgqM849P03lR3tR7qwbjzw%2Fc%2BR%2Bv0oIoHyDgEDrbw%3D.sha256 * Invidious - web interface Youtube browser https://github.com/iv-org/invidious -Various online instances: https://instances.invidio.us/ RSS: This is _much_ faster than youtube-dl on my internet connection, so should really try to use it instead. https://www.youtube.com/feeds/videos.xml?user=FILMAUSTRALIA - Unforunately doesn't work well with some RSS viewers. Nrss only shows headings. - can be sort-of viewed in links, but image links not displayed. YouTube is now using WebP images, and these are supplied by --get-thumbnails. - However JPEGs still available as: https://i.ytimg.com/vi/[VIDEO ID]/hqdefault.jpg -So just use --get-id to grab the Video ID string and put it in.