[HN Gopher] I accidentally started a movement - Policing the Pol...
___________________________________________________________________
I accidentally started a movement - Policing the Police by scraping
court data
Almost 3 years ago, I posted a story of how a post I wrote about
utilizing county level police data to "police the police" to
r/privacy and hackernews.
https://old.reddit.com/r/privacy/comments/gr11aw/i_think_i_a...
The idea quickly evolved into a real goal, to make good on the
promise of free and open policing data. By freeing policing data
from antiquated and difficult-to-access county data systems, and
compiling that data in a rigorous way, we could create a valuable
new tool to level the playing field and help provide community
oversight of police behavior and activity. In the almost 3 years
since the first post, something amazing has happened. The idea
turned into something real. Something called The Police Data
Accessibility Project. (https://www.pdap.io) More than 2,000
people joined the initial community, and while those numbers
dwindled after the initial excitement, a core group of highly
committed and passionate folks remained. In these 3 years, this
team has worked incredibly hard to lay the groundwork necessary to
enable us to realistically accomplish the monumental data
collection task ahead of us. Let me tell you a bit about what the
team has accomplished in these 3 years. Established the community
and identified volunteer leaders who were willing and able to
assume consistent responsibility. -Gained a pro-bono law firm to
assist us in navigating the legal waters. Arnold + Porter is our
pro-bono law firm. -Arnold + Porter helped us to establish as a
legal entity and apply for 501c3 status -501c3 status granted
-We've carefully defined our goals and set a clear roadmap for the
future -Hired first full-time staff. -PDAP was awarded a $250,000
grant by The Heinz Endowments So now, I'm asking for help, because
scraping, cleaning, and validating 18,000 police departments is no
easy task. The first is to join us and help the team. Perhaps you
joined initially, realized we weren't organized yet, and left? Now
is the time to come back. Or, maybe you are just hearing of it now.
Either way, the more people we have working on this, the faster we
can get this done. Those with scraping experience are especially
needed. The second is to either donate, or help us spread the
message. The more donations, the more data we can gather. I want to
thank the r/privacy community especially. It was here that things
really began. TL;DR: I accidentally started a movement from a blog
post I wrote about policing the police with data. The movement
turned into something real because of r/privacy and hackernews:
(Police Data Accessibility Project). 3 years later, the groundwork
has been laid, non-profit established, full-time staff hired, and
$250,000 in grant money and donations so far! Scrapers so far
Github https://github.com/Police-Data-Accessibility-
Project/Scraper... Discord if you would like to join the efforts:
https://discord.com/invite/wMqex8nKZJ *This is US centric
Author : kristintynski
Score : 311 points
Date : 2022-09-19 19:42 UTC (3 hours ago)
| bckr wrote:
| I'm curious if there are opportunities to be a force multiplier
| here. I see that the Readme says "there's no automated scraper
| farm" yet. Getting that set up seems crucial. Will jump on the
| Discord :)
| kristintynski wrote:
| The community has been writing scrapers since the beginning,
| but it's a huge task. We've also written scraper templates,
| since many counties use the same systems.
| https://github.com/Police-Data-Accessibility-Project/PDAP-Sc...
| bckr wrote:
| Yep, I see that there are a lot of existing scrapers. Getting
| those up and running automatically on a backend host will
| make things a lot smoother.
|
| Also, I recommend using Playwright instead of Selenium. I'm
| in the Discord now and will be hanging out, so looking
| forward to chatting more about this and contributing.
| josh-pdap wrote:
| Thanks for taking a look! DM me on Discord if you'd like to
| chat about this.
| motohagiography wrote:
| On the back end, are you using a graph? Having done some public
| sector accountability stuff where the org structures themselves
| were obfuscated, graphs and a clear data model were the decisive
| tech.
| josh-pdap wrote:
| Can you say more about this? Feel free to reach out to my email
| (josh.chamberlain@pdap.io) if you'd like to share more. It
| sounds like you have some expertise that would be incredibly
| useful to us.
| meteor333 wrote:
| Thanks for sharing about your project!
|
| Do you mind giving us brief on what kind of data you are
| collecting and highlight any interesting findings so far?
| contingencies wrote:
| I wonder if it is legal / possible to record police radio traffic
| and associate it with the records?
| getcrunk wrote:
| Alot of comms has moved to being encrypted. Before that yes it
| was considered public
| contingencies wrote:
| You can still get the 'metadata'. ;)
| ejb999 wrote:
| As a long time scanner enthusiast, if you actually spend
| anytime listening to PD radios (which is legal and easy), you
| will be disappointed with how little information actually goes
| out over the unencrypted air - just enough to get units
| rolling, after that, very little, for obvious reasons.
| xen2xen1 wrote:
| Because cops are lazy and want to put it all in a report
| later? I would.
| KennyBlanken wrote:
| A ton of stuff is done on mobile terminals. Computer aided
| dispatch including status and location of a unit, report
| writing, identity checks, etc. Scanners were useful because
| a cop would ask the dispatcher for a license check on Joe
| Blow, age 34, of Smithville - or registration check on a
| certain license plate number. These days by the time an
| officer flips on his lights to pull someone over, they
| probably already know the registration/insurance status,
| license status of the registered owner, their photo, etc.
|
| Then there's email and other messaging for lower priority
| things, and phone calls for stuff that is sensitive.
| daviddever23box wrote:
| Public airwaves FTW.
| curiousllama wrote:
| Really love the idea, and the passion behind it. Def could have
| legs.
|
| Here's the pitfalls I see you falling into:
|
| (1) seriously, what data are you collecting? "Everything" isn't a
| great answer (who's supposed to use 'everything', anyway?
| "Anyone"?). "Apples-to-apples police misconduct statistics" is a
| good one.
|
| (2) it's important to clarify 1 because you need to know who
| you're serving, and why. Different activists need different data.
| "Have all data" sounds good until you need to decide how to
| allocate your resources.
|
| (3) more deeply, data is the land of edge cases. Even just with
| police misconduct, you need to get DEEP to rigorously compare
| seemingly-simple stats like "# of unjustified police killings".
| If you don't start narrow, you'll never show value. If you don't
| show value, nobody will ever care you exist.
|
| When I look at the data you've collected, it ranges from annual
| reports, to municipal contact info, to crime stats. What's
| important to collect at scale? To whom? What do they need it for?
|
| Again - great, ambitious idea! But $250k goes fast. Show value
| before it runs out!
| treis wrote:
| anm89 wrote:
| Thanks for taking the time to be obnoxious to one of the tiny
| percentage of people who's willing to make a personal sacrifice
| towards important real world problems for a reason other than
| making money.
|
| Do everyone a favor and go back to wherever you came from and
| stay there quietly.
| treis wrote:
| hinkley wrote:
| pessimizer wrote:
| > trying to gain followers on LinkedIn instead of doing
| impactful work.
|
| wtf are you smoking? Do you think that activism doesn't involve
| marketing, and do you think that people don't need to market
| themselves in order to find volunteers? Do you call out
| charities for advertising, or call out employment ads for
| sounding like they're desperate for employees?
|
| > You say it's turned into something real but I'm struggling to
| find it in the sea of marketing.
|
| Did you just ignore the URLs?
| llbeandata wrote:
| On their github [0] I found https://app.pdap.io but it is not
| loading for me.
|
| [0] - https://github.com/Police-Data-Accessibility-
| Project/PDAP-Da...
| josh-pdap wrote:
| This was an early experiment, we don't have a published app
| right now.
| kristintynski wrote:
| You can find the roadmap, goals, and a lot of detail on the
| organizational structure and the scrapers that have been
| written so far are here https://docs.pdap.io/activities/our-
| process and here https://github.com/Police-Data-Accessibility-
| Project
| kylewatson wrote:
| With all due respect, you really need to be able to explain
| the goals in a sentence or two, and then follow up with a
| link to the pdf. An elevator pitch, a vision statement, a
| clearly written user statement, all would go a long ways
| towards explaining why this is a worth while use of ones time
| and energy.
|
| I don't know whether the goal is to find bad cops who have
| been fired and get hired one town over and try to prevent
| those hirings? Or if the idea is to find people who have been
| arrested and ruin their lives? Or if the goal is simply to
| make the data available and let people use it for whatever
| they wish, from deepfakes to erotic fan fiction.
|
| Data wants to be free. The data is the end goal. Public data
| should be public, etc all sound great. But, I've seen the mug
| shot database ruin people's lives, and eventually its
| founders had mugshots of their own.
|
| Scraping and publishing is one thing, but knowing the goals
| is even more important because it would let me know why we're
| scraping, what we need to scrape, how to make that available,
| etc.
|
| What problem are you trying to solve? And Why?
|
| Police data is not easily searchable? Okay, so what? What
| good is it to make this available? What uses could it have?
| Even if the goal is just to build it and see how people use
| it, it would be helpful to know that.
|
| I strongly suggest you consider working backwards. It's been
| 3 years. What does 3 years from now look like? What is the
| press release? How do you know you were successful, what does
| that mean?
|
| Availability is nice. It could lead to transparency. That may
| lead to accountability. Is that the goal? Right now this data
| is hard to access. If you succeed it will be easier to
| access. And so what? What changes in the world could come
| because of this? If you fail, what lost opportunity do we
| mourn? You gotta have some sort of easily conveyed reason for
| doing this. If the data accessibility alone is the end goal,
| that's fine, not enough for me, but at least make that clear
| and convince me that's worth the effort.
| ALittleLight wrote:
| I like writing web scrapers and this is an interesting project
| idea. If I understand right you are looking for volunteers to
| write scrapers that would take a police department, scrape the PD
| website, and download any PDFs or documents that gather data
| about the police department. Is that right? If so, I feel that's
| not super clearly communicated - I had to look at a couple
| example scrapers before arriving at this guess.
|
| I do have a few questions too:
|
| 1. Will this scale? One problem with scrapers is that they break
| when people update their website. I'm imagining this problem
| multiplied by 18,000 and compounded by each scraper potentially
| being written by a different volunteer.
|
| 2. Where are the scrapers getting run?
|
| 3. How do the documents that the scrapers collect get transformed
| into usable data?
|
| 4. It seems to me like a scalable solution would be a standard to
| report data, a law to compel police departments to follow that
| standard, and then a system to collect that data and make it
| available. Do you work with police departments at all on data
| reporting?
| josh-pdap wrote:
| 1. I replied to the parent comment here; our answer to the
| scale problem is to recognize that people doing web scraping
| are as decentralized as the police. Our goal is to empower
| people who have questions about the police to answer them.
|
| 2. You can run them locally. We're not running the scrapers
| anywhere, or storing extractions anywhere.
|
| 3. This is a big, big question. Right now, the answer is
| dependent on the use case. Rather than trying to make the
| world's biggest database, we're going to respond to community
| needs and build this kind of thing as it comes up.
|
| 4. https://measuresforjustice.org/ is doing something like
| this! We're interested in creating incentives for police
| departments to make their data more accessible and transparent.
| ALittleLight wrote:
| Not to be too rude/negative/mean - but it seems like a big
| concern to me that a "police data" project that's existed for
| three years, has a couple employees, has some funding, etc -
| doesn't really seem to have any police data. If I wanted to
| write a scraper to gather documents off of police websites
| and run it by myself and store my results locally - I could.
| What does your project add?
|
| What I would expect to see is something like:
|
| 1. Here's what data we want from each police department each
| day. Here's what value you should use to indicate that data
| is not available.
|
| 2. Here's a list of police departments. Write a scraper. If
| it passes tests to show it's generating valid data, and code
| review, we will run the scraper in a daily basis storing the
| output in this database.
|
| 3. Here's how you can query our database.
| josh-pdap wrote:
| It's ok, this feedback is how we understand what our work
| looks like to other people, and how we improve!
|
| We've only had paid staff for about 6 weeks. We've run
| several experiments and started from scratch a few times
| over the years. We've been slowly inching the project
| forward in our spare time; I was the only volunteer for
| much of that time, and I can't even code!
|
| 1. Sounds great, we're building something like this.
|
| 2. Even making a list of police departments is a big
| challenge. We've made a good start but have work to do.
|
| 3. Yep, soon we'll have something to query.
| elicash wrote:
| I'd be interested in helping scrape, but no experience. I'd
| presume every county is different so there's no simple training
| you can put folks through? Other tasks, like monitoring for
| things breaking?
| Jcowell wrote:
| Post like this are interesting because as an idea you would think
| that HN would the best target. Even if no one here provides a a
| single character of code they can provide insight Into pitfalls
| and experiences they've run into when doing this sort of thing. I
| hope the comment section are fortuitous in advice.
| josh-pdap wrote:
| Yep! It's really helpful to see where people find problems and
| want to jump in.
| lazyasciiart wrote:
| Very interesting. I have written scrapers for the jail inmate
| data in the couple of counties nearest me - does that come under
| the scope of what you're doing, or not quite?
| VWWHFSfQ wrote:
| Is it possible to see the data the PDAP has scraped? I visited
| the website but I don't see any actual data.
| josh-pdap wrote:
| We don't have any scraped data yet. I replied to the parent
| post addressing some of this, but mostly if people need the
| data they run a scraper locally and use the data that way. At
| the moment, our energy is going into building an app to help
| people submit and manage our database of data sources:
| https://docs.pdap.io/activities/data-sources/what-is-a-data-...
| VWWHFSfQ wrote:
| Ah very good! This sounds like it would be a perfect
| application of https://datasette.io/
| veritas3241 wrote:
| https://docs.pdap.io/activities/data-sources/explore-data-so...
| It's in the docs.
| debacle wrote:
| This is important. Locally, we had a sheriff who was being
| heavily, heavily criticized due to several deaths at the county
| facility. This was at the height of the protests a few years ago.
|
| It was a lot of work to find data on policing nationwide, because
| the question really was "Is the sheriff doing a bad job, or do
| bad things happen sometimes?"
|
| After some hard work trying to identify other cities with similar
| socioeconomic circumstances and populations, it became clear that
| our local sheriff was actually better than average, and that much
| of the outrage was fabricated.
|
| That's also when I learned that many people don't want to listen
| to statistics unless they agree with their own preconceptions.
| [deleted]
| joshenders wrote:
| Got excited by this call to action and then discovered this filth
| by your executive director in your discord.
|
| What a huge letdown.
|
| Take your pathetic crypto grift elsewhere please.
|
| <<I had a meeting with Dave @ Future Foundation discord:
| https://discord.gg/vHqwbDkX.
|
| I shared that what we need help with most now is expertise to
| help us make quick decisions about what directions to go with
| crypto
|
| I listed the 3 main ways we've thought of for using web3 tools:
|
| 1. DAO for the PDAP scrapers - community ownership
|
| 2. PDAP NFT to incentivize code contributions - contributor
| recruitment
|
| 3. Use distributed ledger tech for data storage - transparency +
| traceability
|
| He said he's going to put together a *brainstorm session just for
| PDAP* with some of the experts who have collected in that
| Discord! They will help us answer our foundational questions and
| move us along the path to making stuff.>>
| josh-pdap wrote:
| The post you quoted lists _potential avenues for using web3
| tools_. You may notice we have not gone down any of those
| avenues. We aren 't doing crypto stuff.
|
| We did indeed spend some time looking into web3 and peer-to-
| peer, and ultimately decided it's not for us. Most of the
| energy there is spent trying to make money / scam people. Peer-
| to-peer is cool too but has its own risks.
|
| Thanks for checking out the discord, though!
| bb88 wrote:
| Yeah, I'm glad you went this route.
|
| It's one thing if you want to have a crypto wallet to accept
| donations -- that's not that controversial. It's another to
| drink the web3 kool-aid and base your organization's future
| on that.
| kristintynski wrote:
| PDAP is not crypto based, and there are no plans to make it so.
| The message you pasted here was about a meeting taken last year
| with someone who was proposing this. It isn't something the
| community or leadership was interested in. PDAP is a non-
| profit, not a crypto startup.
| dang wrote:
| Yikes. Other users have pointed out a bunch of things already
| but I need to add that this sort of name-calling and personal
| attack* is against the rules and spirit of this site, and is
| the sort of thing we ban accounts for. Regardless of how right
| you are or feel you are, please don't do it again.
|
| If you'd please review
| https://news.ycombinator.com/newsguidelines.html and stick to
| the rules when posting to HN, we'd appreciate it.
|
| * especially against a new user--greeting newcomers with a
| torrent of abuse is really bad
| jovial_cavalier wrote:
| Floating the idea of having a DAO governance system =/= "crypto
| grift." It's a legitimate way of trying to ensure that control
| of a distributed system remains distributed. It doesn't even
| sound like they've pursued the idea. You can, in theory, have a
| non-profit DAO.
|
| Recently there has been an unreasonable amount of hostility
| towards anything that even mentions crypto or any related
| technology. I wonder what the source of it is.
| yuan43 wrote:
| I don't understand the claim you're making in this response.
| Can you condense it to 1-2 sentences (and cite source w/ link
| if available)?
| ooouups wrote:
| josh-pdap wrote:
| I think if a project like ours ever did use blockchain, it
| would be behind the scenes as a part of the product;
| decentralization and transparency are key parts of our ethos!
| However, it's too unstable and as you can see by the
| comments, mentioning "web3" costs a lot of credibility in
| many, or maybe most, communities online.
|
| https://knish.io/ is a good example of people using
| blockchain to make real products. I don't know that it's
| ready though.
| multjoy wrote:
| socialismisok wrote:
| I was so worried that the "filth" you were pointing to was
| going to be holding police accountable. Glad to see it is
| actual filthy web 3 garbage.
| dang wrote:
| Could you please avoid flamewar comments on HN and, also,
| please make sure you aren't using this site primarily for
| political or ideological battle? That's one line at which we
| ban accounts (https://hn.algolia.com/?sort=byDate&dateRange=a
| ll&type=comme...). Regardless of what they're battling for or
| against, it's not what this site is for, and it destroys what
| it is for.
|
| If you wouldn't mind reviewing
| https://news.ycombinator.com/newsguidelines.html and taking
| the intended spirit of the site more to heart, we'd be
| grateful.
| ben174 wrote:
| could you elaborate exactly where the filth is here? I think I
| get it, but I'm not super-well versed enough to understand what
| the motive would be here.
| ALittleLight wrote:
| Edit: Seems like this criticism doesn't apply to this
| project. I think that's good. What I wrote below is just an
| explanation of why someone might view crypto as a red flag in
| a charity project.
|
| ---
|
| I think the basic idea is that this web3 crypto stuff is
| pretty scammy. It would be like finding out the not-for-
| profit you were thinking about working with also sells
| timeshares - maybe it is, against all odds, legit, but still
| not a good look and kind of a red flag.
|
| Another take on it is - if I want to volunteer at a place I
| don't expect to get paid. If I'm working for payment, I don't
| want my payment to be in NFTs. So, the web3 intrusion into
| this idea is unnecessary and doesn't fit for either
| volunteers or employees.
| josh-pdap wrote:
| That's one of the many reasons why we abandoned the crypto
| idea after some initial optimism and research.
| kristintynski wrote:
| Nothing about PDAP is web3, or crypto based, though some
| community members have suggested it (especially last year
| among all the Web3 hype.) Turning into a web3 company is
| not happening.
| wongarsu wrote:
| It certainly set off my alarm bells for people who try to do
| pointless blockchain stuff for personal profit (or I guess
| fun and street-cred/CV-lines).
|
| That doesn't mean it has to be the case here. But at first
| glance a DOA seems more like a detriment here (police can
| outspend citizens) and NFTs are NFTs, no explanation needed I
| suspect. Suggestion 3 might have merit, but storing the data
| in the blockchain (instead of just some hashes for
| timestamping) makes it look like some overambitious vanity
| project again.
| josh-pdap wrote:
| Hello! I'm the executive director. I have a design background,
| have done product management in the past, and aside from keeping
| the lights on at PDAP and making sure we're tax-compliant I am in
| a product role. I talk to people using police data, and figure
| out where we can add value to make the data more accessible.
|
| TL:DR; If you want to write scrapers: go for it! Run your
| scraper, share the results in Discord and with your friends, and
| talk about the process. We'll be listening, and it will help us
| build tools to support this important work.
|
| A few things to clarify:
|
| a. The source of truth for "what are we doing right now" and "how
| can I contribute" is https://docs.pdap.io/.
|
| b. Empowering people who write scrapers is a part of our broad
| mission of "police data accessibility", but we have some
| foundational work to do first! Right now our primary project is
| creating a database of police agencies and data sources. This
| will help people understand what kinds of data are available, at
| which agencies, with which steps to access it. It will also help
| us create archives of the primary sources, so that if they get
| taken offline we can still go back and scrape them.
|
| c. What we have realized in the past few years: there are already
| a ton of people writing and using web scrapers for their day to
| day work. They are as decentralized as our police system. Our
| scrapers repo will reflect that. We shouldn't all rely on one
| library, or even one language. The people who need the data are
| most motivated to maintain scrapers, and we expect that
| maintenance will be ad-hoc and as-needed for the immediate
| future. In most cases, data already published on the internet is
| useful to local users as-is.
|
| d. If you have a question you'd like to answer about the police,
| here's the investigation process:
|
| 1. Determine whether public data exists to answer your question.
| Use google to find the appropriate agency, and see what they're
| publishing. 2. Determine how it can be accessed; do you need to
| make a FOIA request? Is there a URL? 3. If there's a URL,
| determine whether you need to write a scraper to access the
| records. Often, the records can simply be downloaded. 4. Write
| and run a scraper, if you need one! 5. If there's not a URL, make
| a records request for the public information. This is a long and
| complicated process. 6. Share the data with your friends.
|
| This means that scrapers are helpful and necessary some of the
| time; but not always, and not as the first step. We're trying to
| help with steps 1, 2, 3, 5, and 6. The theory is that writing
| scrapers is something people can easily slot in and help with;
| and that, depending on what question you're trying to answer, two
| scrapers for the same data source might look wildly different.
|
| Scrapers are an important part of the ecosystem, but they're one
| piece of the puzzle.
| Merciernmon wrote:
| Josh, many of your comments here are displaying as "dead." The
| HN FAQ[0] says:
|
| > What does [dead] mean?
|
| > The post was killed by software, user flags, or moderators.
| Dead posts aren't displayed by default.
|
| I suspect that it is a false positive. Maybe email the mods for
| help? hn@ycombinator.com
|
| [0] https://news.ycombinator.com/newsfaq.html
| josh-pdap wrote:
| Thanks! I contacted them and I've been reinstated.
| icelancer wrote:
| Not sure why this got killed (dead in HN terms) but I vouched
| for it.
| soco wrote:
| Half of josh's comments show up as dead, so maybe you and
| others who can, look around and vouch for them as well.
| There's something nefarious going on...
| sophacles wrote:
| Over half this person's comments (the Josh-pdap person I
| mean) have been flagged.
|
| Not sure if this is some false positive on anti-spam (new
| account, lots of comments in a short time on a single story)
| or if it's someone with a vendetta. @dang this is a little
| weird...
|
| (I know, this site doesn't have magic @, but also i think the
| site moderator probably has scripts searching for
| mentions/summons anyway. I would anyway...)
| dang wrote:
| The comments weren't flagged. HN's software has other kinds
| of filters too, and comments by brand new accounts are
| treated a little more skeptically. Fortunately users had
| vouched* for all but two of josh-pdap's comments by the
| time I saw them. I restored the other two and marked the
| account legit.
|
| * https://news.ycombinator.com/newsfaq.html
| josh-pdap wrote:
| What do you mean? I can still see it, but you can't?
| jrockway wrote:
| Yeah, all of the new comments from the account are dead. I
| think creating a new account, posting an article, and posting
| a handful of comments looks a lot like spam to HN. I've
| vouched for the comments, but would definitely recommend
| emailing the admins.
|
| Josh, I can't reply to you directly, but yeah... you can see
| the comments, but most people can't, unless they have enabled
| "showdead" in their preferences.
| josh-pdap wrote:
| Thanks! It wasn't me who posted the article, but I agree
| with the theory that it's likely some well-meaning "new
| account spam prevention" thing.
| thejqs wrote:
| Thank you.
| account-5 wrote:
| Apologies for my ignorance but how is this going to police the
| police? I read the original blog post, there was lots of
| inferences/could and might be's/etc made but little in the way of
| proof of anything. What's to stop the police saying it was just
| circumstance that provided your results?
|
| I'm not here defending the police, or denigrating the project,
| just playing devils advocate. What happens if the police just
| ignore you?
| josh-pdap wrote:
| Aren't forums like this for devil's advocating, like, almost
| exclusively? Working as expected!
|
| We've come a long way since that post in terms of strategy and
| focus. Most of that time was spent with between 1 and 3
| volunteers, working a couple hours a week.
|
| Transparency is a good goal in itself, I think. People are
| already _using_ this public data, we 're just trying to make it
| more accessible.
|
| "Policing the police" was the original phrase used on reddit,
| but if you look at our website (https://pdap.io), that's not a
| phrase we use.
| vmh1928 wrote:
| Assuming the data is accurate it can be used to show
| disparities between groups for a variety of situations -
| traffic stops, arrests, jail vs. diversion programs, charge
| stacking, etc..
| lazyasciiart wrote:
| The goal of projects like this (I have no contact with this
| one) is usually to convince politicians and/or the public of
| their results, and those groups are the ones to actually push
| change.
| Pelerin wrote:
| Thank you for your work with this! One question I have:
|
| You say in your FAQ "We aren't a watchdog--our activism is data
| collection and accessibility, not analysis or research."
|
| Can you note any instances of other people using your data for
| analysis or research?
| josh-pdap wrote:
| We're still developing those relationships, and we haven't
| generated any novel data that is deeper than web URLs. I'm
| based in Pittsburgh so we're still working with local
| journalists, activists, etc. to understand how they use the
| data and how we can help.
| anamexis wrote:
| Just a heads up that your comments are getting marked as dead
| (shadow banned). You seem to have fallen afoul of some HN
| spam trap for new accounts.
| ben174 wrote:
| a while back I created www.bartcrimes.com to publish police
| reports which were intentionally hidden behind a mailing list you
| must get approved to be a member of. Turns out, the public loves
| this kind of thing.
| abeppu wrote:
| That's cool but why are the most recent entries from Sept 2021?
| Did BART do something even more effective to stop these updates
| from getting out?
| josh-pdap wrote:
| I love that you wrote it on BART. I spent my year of BART time
| solving chess puzzles.
|
| "Making public information public" is a good tagline too.
|
| Do you know what kinds of work people did with the data? It
| seems to me one of the best ways to address BART crime would be
| to support the impoverished and desperate people who don't have
| any recovery or mental health support, but that work is slow...
| RobertRoberts wrote:
| Is their any data sources we could scrape to stop crimes in our
| neighborhoods so the police don't have any reason to come around
| and cause problems?
| kennywinker wrote:
| You could scrape improved social safety nets, better access to
| birth control, and taxing the rich to pay for it.
| RobertRoberts wrote:
| How could we use that data to help prevent crime?
|
| To best mimic the success of the original post, it would be
| good to be able to identify people doing the most damage to a
| neighborhood so we could help them first with tax relief,
| food, birth control, whatever would help the most.
| [deleted]
| [deleted]
| steve76 wrote:
| quickthrower2 wrote:
| It would be interesting to do this too. Although this is going
| to be a yak shave (a good one) - changing the upstream things
| that cause people to turn to crime.
| josh-pdap wrote:
| https://raheem.ai/ is an interesting project. One idea someone
| in one of their community calls had was scraping dispatch data
| to figure out where social services might be useful, and
| they're creating a dispatch app which lets people ask for non-
| police help.
| celestialcheese wrote:
| For folks who do this kind of disparate data-source scraping at
| scale, what does best practices look like? What kind of tools are
| used in industry?
|
| Maintaining scrapers for 18k county websites and PDs is no small
| task and looking through the docs for PDAP, it seems like this is
| still a very open question.
| enviclash wrote:
| I would like to research on the data, is it available as a
| source? (Email in profile)
| steve76 wrote:
___________________________________________________________________
(page generated 2022-09-19 23:00 UTC)