an integration between saved reddit posts and pinboard
an integration between saved reddit posts and pinboard.
## Outline:
### Getting data from reddit
Currently i'm getting only the first page of saved, no NSFW items
specifically, comments and posts saved to your reddit user profile can be pulled into pinboard through this project. the following information is pulled down and used when pinning to pinboard:
- subreddit (becomes a tag)
- post title (becomes title)
- post description, if it exists (becomes description)
- post url (becomes url)
the tag "added-by-pynnit" is also added to each entry moved to pinboard in this way. This allows for easy viewing of all imported links. I found this very useful when I was writing the script in the first place, but you may not want it.
## Using this bullshit
You need several things to use this project:
- Reddit username
- Reddit password
- Reddit client ID
- Reddit client secret
- Pinboard api key
Your reddit un/pw you should already have. The client ID and client secret can be generated by following the instructions here:
Your pinboard api key can be found here (assuming you are logged in):
https://pinboard.in/settings/password/
You need to export these in your shell in the following format:
```
export REDDIT_UN=''
export REDDIT_PW=''
export REDDIT_ID=''
export REDDIT_SECRET=''
export PINBOARD_TOKEN=''
```
Then run the following commands, in the order given:
```
python main.py
python pinboard.py
```
Pulling reddit posts through the main.py file is very fast. The pinboard step, however, is quite slow. The API docs published by pinboard require that you only make a call once per 3 seconds, so if you're adding a lot of entries at once (say, during the intial move over) this can take a while, but its still fast enough for my purposes.
## Information / my shit
### Outline:
#### Getting data from reddit
- [X] refactor to use praw instead of requests
- praw has more functionality more obviously than I can figure out with requests.
- its aggrevating.
@ -15,28 +54,26 @@ Currently i'm getting only the first page of saved, no NSFW items
- each one will have different fields but REDDIT DOESN'T DOCUMENT THIS
- because reddit is stupid, that's why, i guess.
### Parse data
Do i need to do anything here, actually, or is json fine?
### Putting data in pinboard
#### Putting data in pinboard
So far i've done nothing.
- [X] Get auth token to work
- Finally got this to work; I had a fundamental mistunderstanding of what pinboard meant by "method" in the URL.
- [X] Figure out how to pull existing posts
- [] Figure out how to post an item to my feed as public
- [X] Figure out how to post an item to my feed as public
- [ ] Figure out how to post an item to my feed as private
- [ ] Enable a conditional; NSFW items get posted as private, regular items as public.
- [] Iterate through a list.
- [X] Iterate through a list.
### IF WE RECEIVE OVER 5 MILLION DOLLARS I WILL:
- [] figure out how the fuck to compare urls/titles against already existing entries in pinboard
#### IF WE RECEIVE OVER 5 MILLION DOLLARS I WILL:
- [X] figure out how the fuck to compare urls/titles against already existing entries in pinboard
- and obviously don't add dupes
- HAH you don't have to worry about this, Pinboard will do this for you! Hurray.
- [ ] figure out how to pull in RES saved items; that'll be weird slash impossible maybe
## Information about reddit (i'm so sorry)
### Information about reddit (i'm so sorry)
So, there are multiple kinds of reddit posts, and each kind of reddit post seems to have distinct names for the same things, which is REALLY fucking annoying. Its extra frustrating because there's not just a quick lookup for this, you have to just dig through Too Much json.
If you're using PRAW, a reddit /post/ has an attribute called `.is_self` that's boolean. If its true, its a text only post, if its false then its a link post.
Reddit comments do not have this attribute. They DO have an attribute called `.is_root`, which i use to differentiate themm.
Reddit /comments/ do not have this attribute. They DO have an attribute called `.is_root`, which i use to differentiate themm.
