Tag Archives: command line

Scrape Keywords from Indeed.com Job Postings











Job Posting Crawler

This is code that will pull each job posting for a specific job title in a specific location (or Nationally) and return / plot the percentage of the postings that have certain keywords. The code is set up to search for all words except stopwords, and other user-defined words (there is probably a much more efficient way of doing this, but I had no need to change this once I had the code running). This allows the user to see common technical skills, as well as common soft skills that should be included on a resume.

NOTE: I got this idea from https://jessesw.com/Data-Science-Skills/. Obviously, just using his code would be of no real benefit to me, as I wanted to use the idea to help better my skills with scraping data from HTML files. So, I used his idea and developed my own code from scratch. I also modified the overall process a bit to better fit my needs.

NOTE2: This code will not be able to identify multiple-word skills. So, for example, ‘machine learning’ will show up as either ‘machine’ or ‘learning’. However, ‘machine’ could show up for other phrases than ‘machine learning’.

To run the code, change the city, state, and job title to whichever you wish. After generating the plot, you might need to add ‘keywords’ to the attitional_stop_words list if you do not want them to be included.
Continue reading Scrape Keywords from Indeed.com Job Postings

How to: Passwordless SSH

As some of you know, I prefer to set up passwordless logins to all of my accounts on remote machines. I recently made a post describing how to enable passwordless SSH to compute nodes, however what if you are attempting to enable passwordless logins to remote machines?

If you are on a Linux machine, or have a copy of the “ssh-copy-id” script on your system then the process is fairly simple.  You must first create the private/public key pairing.  For passwordless SSH, just accept the defaults for each option.

ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/cmaqadj/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/cmaqadj/.ssh/id_rsa.
Your public key has been saved in /home/cmaqadj/.ssh/id_rsa.pub.

Continue reading How to: Passwordless SSH

How to Fix “Output Conversion Error”

As part of my research for my Ph.D. I am on a team that is currently developing an adjoint of the EPA’s CMAQ air quality model.  In the process of integrating all parts of the model into the full adjoint model, I ran into an error that was rather difficult to resolve.

Running the model would result in many occurances of the following error:

forrtl: error (63): output conversion error, unit -5, file Internal Formatted Write
Image              PC                Routine            Line        Source
ADJOINT_FWD        00000000009B34BD  Unknown            Unknown     Unknown
ADJOINT_FWD        00000000009B1FC5  Unknown            Unknown     Unknown
ADJOINT_FWD        0000000000969210  Unknown            Unknown     Unknown
ADJOINT_FWD        000000000092AADF  Unknown            Unknown     Unknown
ADJOINT_FWD        000000000092A312  Unknown            Unknown     Unknown
ADJOINT_FWD        000000000095305A  Unknown            Unknown     Unknown
ADJOINT_FWD        00000000005D9F94  ckdesc3_             138       ckdesc3.f
ADJOINT_FWD        00000000005A9FD1  open3_               216       open3.F
ADJOINT_FWD        000000000047B395  chk_files_impl_mp    170      CHK_FILES_IMPL.F
ADJOINT_FWD        0000000000485060  chk_files1_mp_chk    347       CHK_FILES.F
ADJOINT_FWD        00000000005666CB  vdiff_               369       vdiffacm2.F
ADJOINT_FWD        0000000000496B7E  sciproc_             228       sciproc.F
ADJOINT_FWD        000000000048DDB5  MAIN__               205       driver_fwd.F
ADJOINT_FWD        0000000000404A1C  Unknown            Unknown     Unknown
libc.so.6          0000003FE9E1D994  Unknown            Unknown     Unknown
ADJOINT_FWD        0000000000404929  Unknown            Unknown     Unknown

     >>> WARNING in subroutine CRTFIL3 <<<
     Error creating netCDF variable for file ADJ_VDIFF_CHK
     Illegal data type    0

     *** ERROR ABORT in subroutine CHK_FILE_CREATE_
     Could not open ADJ_VDIFF_CHK file
     Date and time  13:00:00  July 22, 2001   (2001203:130000)

I spent a lot of time searching online, however I was unable to find a solution for my problem.  After days of debugging, I finally found the source of the problem.

Continue reading How to Fix “Output Conversion Error”