Scrape Keywords from Indeed.com Job Postings

Job Posting Crawler

This is code that will pull each job posting for a specific job title in a specific location (or Nationally) and return / plot the percentage of the postings that have certain keywords. The code is set up to search for all words except stopwords, and other user-defined words (there is probably a much more efficient way of doing this, but I had no need to change this once I had the code running). This allows the user to see common technical skills, as well as common soft skills that should be included on a resume.

NOTE: I got this idea from https://jessesw.com/Data-Science-Skills/. Obviously, just using his code would be of no real benefit to me, as I wanted to use the idea to help better my skills with scraping data from HTML files. So, I used his idea and developed my own code from scratch. I also modified the overall process a bit to better fit my needs.

NOTE2: This code will not be able to identify multiple-word skills. So, for example, ‘machine learning’ will show up as either ‘machine’ or ‘learning’. However, ‘machine’ could show up for other phrases than ‘machine learning’.

To run the code, change the city, state, and job title to whichever you wish. After generating the plot, you might need to add ‘keywords’ to the attitional_stop_words list if you do not want them to be included.
Continue reading Scrape Keywords from Indeed.com Job Postings

Backup with rsync using SSH Tunneling

For those of you that read my blog often, you know that I admin the cluster that our research group uses here at CU Boulder.  Because of this, I get a lot of questions from users who don’t want to take the time to solve their own problems.  Fairly recently, our RAID-6 crashed (we had a 4th drive die and had to rebuild the array).  Normally this wouldn’t be very much of a problem as most of the files saved on our storage drive are just input files that we can re-download from a separate server, or so I thought.  Personally, all my source code is in my home folder, backed up on our data server, and backed up onto my personal laptop.  For researchers in our group who are developing code, not having a backup of source code can lead to many many months of lost work.  Well, as it turns out, many of the people in our group had their source code on our data server (the one that crashed), without a backup anywhere.  So months of work had been lost.  Well, after the rebuild I have gotten many questions on how to set up an ssh tunnel so that they can backup from our cluster, through the front end, to their home computer.

Quit Terminal.app after closing last window

I use Terminal.app on a daily basis, mostly for my research as I am currently building an adjoint of the CMAQ model.  I like to have 4+ windows open at all times while working in Terminal, as it allows me to watch the progress of a simulation while also editing files, compiling, etc.  I have macros setup that will log me into and out of each terminal window.  One of the frustrating things that I ran into was that, after the macro logged me out of the final window, I wanted the Terminal application to quit.  Now, I very easily could have added the Quit Application command to the macro, however I was hoping for a solution that would also work even if the macro wasn’t called.  I surfed the web for a while and found nothing, until I got a response on my post on Apple’s forums.

I am copying te response to this website, however all credit for the material goes to François J. Perreault, who answered the question.

Here’s how to have Terminal quit automatically after closing all your shells:

1. Create a new text file in your home folder named “autoQuitTerminal.scpt“:
tell application "Terminal"

--If there is only one tab remaining, and it contains
--the word "logout" then this is the final window
if (count of (tabs of (every window whose visible is true))) = 1 then
try
set theContents to words of ((contents of tab 1 of window 1) as Unicode text)
set exitLastTab to (theContents contains "logout")
on error
set exitLastTab to false
end try

if exitLastTab is true then
quit
end if

else if (count of (tabs of (every window whose visible is true))) < 1 then
--If no window remains open, then obviously we can quit the app.
--This would occur when the final window is closed without ‘exit’
quit
end if

end tell

Chrome: Unable to scroll while page loads (SOLVED!)

Solution: If you don’t wish to read the entire post, the solution I found was to disable Adblock+ in Chrome.

Over the past few months I have gotten extremely frustrated with the fact that I am unable to scroll a page while any page loads (in the foreground or background, separate tabs, etc).  After all pages have loaded, scrolling works flawlessly.  However, not being able to scroll while a page loads is by far the most frustrating thing I have ever experienced when surfing the internet.

Well I spent quite a bit of time looking around online for solutions to the problem.  Google’s Q&A site suggested disabling some firewall functionality on Windows; being on Mac, this didn’t help me at all.  I finally decided to switch to a separate Chromium based web browser, SRWare Iron.  SRWare Iron gives you all the benefits of google’s Chrome browser, without the privacy concerns.  Some of the differences include the lack of an installation ID, does not provide search suggestions (search data not sent to google), etc.  For a full list of the differences, reference SRWare’s website. Continue reading Chrome: Unable to scroll while page loads (SOLVED!)

Creating Workspaces in Terminal

If you have read some of my other posts, you know that I like to have multiple terminal windows open whenever I am working.  This used to involve opening the Terminal app, and then opening multiple more windows.  Then I had to drag each window into the location that I wanted it to be at.  I later realized that I could have the Terminal app save my open windows as a Workspace, and could define a default workspace to be opened whenever Terminal is opened.

This is rather simple to do, and can be accomplished by following the steps below: Continue reading Creating Workspaces in Terminal