[HN Gopher] Show HN: Search your Twitter history with Algolia
___________________________________________________________________
Show HN: Search your Twitter history with Algolia
Author : transitivebs
Score : 93 points
Date : 2020-04-11 16:40 UTC (1 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| jchook wrote:
| Twitter only stores your last 3200 likes.
|
| What a bummer. I like a lot of very funny or informative stuff on
| twitter and the tweets often pop into my head even years later.
| Sometimes I wanna find that tweet (eg to share) and I can't.
|
| I started using pinboard.in, which archives all my likes, but the
| search suuuucks.
|
| So, I started building my own archive tool, but the search part
| is still hard.
|
| I have looked into annoy, perkeep, tf-idf, Doc2Vec, and various
| other tools, but nothing really works as well as I had hoped.
|
| Your project definitely gave me some ideas! Thanks.
| pgt wrote:
| Wow, I did not know that. This is such a no brainer paid
| feature for a premium Twitter plan: storage & search.
|
| I suspect they probably keep all likes, but do not expose them.
|
| Another premium service that searches your Twitter like is
| https://www.favourites.io/, but it's too costly for me. I might
| pay $10 for a yearly subscription, but not 29 GBP/year, which
| is ~10% of my monthly rent.
| Kinrany wrote:
| If they do keep all the data, you should be able to get it
| via GDPR
| transitivebs wrote:
| Yeah, you can access all the data of your own account, but
| third-party apps accessing your data via OAuth is another
| issue.
|
| In the case of Twitter's API, third-party apps can access
| all of your history if you upgrade to a paid partnership
| plan.
| dewey wrote:
| > I started using pinboard.in, which archives all my likes, but
| the search suuuucks.
|
| What's also annoying is that I don't think there's a way to
| exclude from_twitter bookmarks in Pinboard when you search. So
| now that I enabled this feature a while ago everything is
| cluttered with tweets and Pinboard become useless for me.
|
| Something like a -from_twitter search would be helpful but
| there's only a "+" search right now.
| abc126589 wrote:
| You can access all your favorites (and tweets and DMs) by
| downloading your twitter archive.
|
| You can use Twitter Archive Eraser to browse the whole archive,
| search by keywords, regular expressions, mentions, tweet type,
| media etc.
|
| Twitter Archive Eraser allows you to bulk delete thousands of
| tweets at once also.
|
| https://martani.github.io/Twitter-Archive-Eraser/
| simonw wrote:
| My https://github.com/dogsheep/twitter-to-sqlite tool has a
| solution for that: you can export your entire Twitter account
| as a zip, load more than 3,200 liked tweet IDs from that
| archive, then backfull their full inflate versions by fetching
| them from the API.
|
| Documentation for doing that here:
| https://github.com/dogsheep/twitter-to-sqlite#importing-data...
| bastijn wrote:
| You should put a license in that repo. Witbout a license, the
| default copyright laws apply, meaning that you retain all rights
| to your source code and no one may reproduce, distribute, or
| create derivative works from your work. If you're creating an
| open source project, it is strongly encouraged to include an open
| source license.
| stevenpetryk wrote:
| Seems this was posted primarily to market Saasify, so I'll take
| the bait: pretty cool that this was set up so fast. You've built
| a neat thing, Travis :)
| transitivebs wrote:
| Hahaha you got me :) Thanks, Steven!
| imdsm wrote:
| But hey, Saasify looks neat!
|
| I'm interested for sure
| nzeeshan wrote:
| This is a neat feature! Kudos How about allowing to add other's
| twitter handle and searching their TL?
| Timac wrote:
| If you want to monitor keywords on Twitter and not just in your
| timeline, you might be interested to check the iOS app Clatters
| : https://apps.apple.com/us/app/clatters/id1480930237 Full
| disclosure : I'm the developer
|
| That being said, I am really impressed by this new service.
| Creating such a solution in just a couple of hours/days is
| astonishing.
| transitivebs wrote:
| Thanks, Timac.
|
| You can read more about how I built Twitter Instant Search
| here: https://www.indiehackers.com/post/launched-a-solid-
| saas-in-8...
| transitivebs wrote:
| Yeah, I really like this idea.
|
| Given that the initial version was built in 8 hours, I'm sure
| there are a lot of features that I'll be adding over the next
| few weeks :)
|
| I'm thinking this would make sense as a paid feature?
| iamaelephant wrote:
| You mean that insanely shitty tool that powers HN search? No
| thanks.
| sneak wrote:
| Strange to think that building an actual web spider to make an
| independent index of public tweets is probably illegal these
| days.
|
| This is basically a frontend to Twitter's own datastore via the
| API.
| transitivebs wrote:
| Yeah; it's just strange that this sort of thing is necessary
| given the resources that Twitter has available.
| gge wrote:
| Can I use this to search other peoples tweet history?
| harlanji wrote:
| Nice job. Had a similar idea after seeing the same Tweet, but
| based around importing the downloaded data archive, as I'm
| working with that myself now. Open source, so I might see if I
| can weave it in when I get a chance, if nobody else does it
| first.
| transitivebs wrote:
| Yeah, I'm definitely down to collaborate -- feel free to reach
| out via email and I'm happy to share any profits if you add
| features to the open source repo.
| anw wrote:
| I'm very happy to see a tool like this that can be self-hosted.
|
| Back around 2010 there was a company called Topsy[0] who was
| supposed to be the "Google of Social Media". It would index
| Twitter tweets and Google+ posts. The idea was you could read
| discussions over the years and find trends that could be used for
| research, analytics or marketing.
|
| Sadly, it was bought up by Apple who shut it down a year or so
| later. There is still a giant hole left around search for Social
| Media, with each of them being their own walled garden.
|
| [0] https://en.wikipedia.org/wiki/Topsy_Labs
| transitivebs wrote:
| The idea for this product came last night from Guillermo Rauch
| tweeting that he wished there was a service to search your
| personal Twitter history (tweets, likes, retweets, filters, etc)
| using an Algolia-type search UI (super fast search as you type).
| A whole bunch of people echoed that they'd pay for this type of
| product.
|
| So I decided to build it overnight and open source it.
|
| The majority of the functionality is handled by Algolia and
| Saasify which is the only reason this was possible to build so
| quickly.
|
| Would love to hear people's thoughts - thanks!
| h43z wrote:
| I use the standard twitter search a lot to find some of my own
| tweets. Mostly a "from:@h43z term" does the job for me. Though
| it's not instant twitter provides many useful filters you may
| want to check out
| (https://twitter.com/h43z/status/1234614509400215554)
| transitivebs wrote:
| Will definitely look into adding support for some of these
| common filters.
|
| Thanks for the heads up!
| 101008 wrote:
| Does anyone know a tool to download all tweets from an account in
| JSON format? No need to include likes or RTs. In fct, better if I
| can avoid those.
| simonw wrote:
| If it's your own account the API will let you fetch everything.
|
| If it's someone else's account the API limits you to around
| 3,200 tweets.
|
| I built https://github.com/dogsheep/twitter-to-sqlite tool for
| archiving my own tweets. I've used it to pull tweets from other
| interesting accounts too.
| dorkinspace wrote:
| You could make a tool pretty easily using Twitter's api:
|
| Get started here:
| https://developer.twitter.com/en/docs/tweets/curate-a-collec...
| transitivebs wrote:
| Unfortunately, Twitter limits the number of historical tweets
| they allow access to from their API (3200 iirc). You have to
| pay for a partner account in order to have full API access.
___________________________________________________________________
(page generated 2020-04-12 23:00 UTC)