Forem: matoval

Using Python to Find Jobs on Craigslist

matoval — Tue, 04 Aug 2020 19:42:05 +0000

The problem

I have ran out of jobs to apply for trying to break into tech. I applied to all the remote jobs on LinkedIn, AngelList, We Work Remotely and countless other jobs boards. I had to find a new source for remote job listings, so I started looking on Craigslist. Having to search every city in the US is very tedious and easy to lose track of cities.

My solution

I have played around with Python a little in the past but never built anything with it. Python works very well for web scraping, so I decided to give it a shot.

First I got a list of all the Craigslist city boards urls from Craigslist. Then I used the urlopen function from urllib.requst to read the html from the page and used BeautifulSoup to work with the page data. I got all the city pages urls by getting all the pages "li" tags, then getting the "a" tags from that. Finally getting "href" urls and adding them to a list. I had to remove the first three because they were not city urls.

import csv
from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup

## get all US cities
cities_list = 'https://geo.craigslist.org/iso/us'
uClient = uReq(cities_list)
cities_html = uClient.read()
uClient.close()
cites_soup = soup(cities_html, "html.parser")
cites_li = cites_soup.findAll("li")
cites_urls = cites_soup.findAll("a")
city_url_list = []
##loop through and get url for each city
for a in cites_urls:
  cites_url = a['href']
  city_url_list.append(cites_url)


city_url_list.pop(0)
city_url_list.pop(0)
city_url_list.pop(0)

Now I have all the US cities' urls in a list in memory. I can use them to get the urls for job posting from each city's "web/info design" job board. There will be thousands of jobs so I stored them in a txt file with a url per line.

First I loop through the "city_url_list" list and use urlopen to read the html and BeautifulSoup to parse it and get all the "a" tags with the class of "result-title" which is all the job posts in the "web/info design" board for that city. Then we loop the "job_titles" and add the urls from each job post to a new list. We then write the list from each iteration to the txt file with each url on its own line.

for url in city_url_list:
  my_url = '{}/d/web-html-info-design/search/web'.format(url)
  #get data from url
  uClient = uReq(my_url)
  page_html = uClient.read()
  uClient.close()
  #parse html
  page_soup = soup(page_html, "html.parser")
  #grabs each job title
  job_titles = page_soup.findAll("a", {"class": "result-title"})
  jobs_url_list = []
  for j in job_titles:
    # print(j['href'])
    if j['href']:
      jobs_url_list.append([j['href']])
  file = open('jobs.txt', 'a', newline='')
  with file:
    write = csv.writer(file)
    write.writerows(jobs_url_list)

Now we have a txt file with over 2000 job posting urls that may be developer jobs but we want to filter through those job postings and find only jobs that have the word developer in the title and/or description.

First we need to get all the urls from the txt file and add them to a list. Now we have a list in a list, so we assign the first list in the list to a variable. Now the variable "job_urls" can be iterated through and we can again use urlopen to read the html and BeautifulSoup to parse it. Then we need to check if any of the posts have been deleted by looking for a "span" tag with the "id" of "has_been_removed". If it has been removed we move on to the next url, if the post has not been removed we continue on. Now we look for the title and description so we can find the word "developer". We get the "span" tag with an "id" of "titletextonly" which is the title and we also get the "section" tag with an "id" of "postingbody" which is the description. We check to make sure there is a value for both the title and the description, and if there is one we continue. Then we check if the title or description has the word "developer" in it, if it does we add the url for that job post to a list and write that list to a txt file that will have a url per line of only developer jobs.

import csv
from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup

with open('jobs.csv') as f:
  l = list(map(str.split, f.read().split('\n\n')))

## get only developer jobs
job_urls = l[0]
for j in job_urls:
  filtered_list = []
  uClient = uReq(j)
  job_html = uClient.read()
  uClient.close()
  job_soup = soup(job_html, "html.parser")
  job_removed = job_soup.find("span", {"id": "has_been_removed"})
  if job_removed == None:
    job_title = job_soup.findAll("span", {"id": "titletextonly"})
    job_body = job_soup.findAll("section", {"id": "postingbody"})
    if (len(job_title[0]) != 0 and len(job_body[0]) != 0):
      if (job_title[0].text.find("developer") != -1 or job_body[0].text.find("developer") != -1 ):
        filtered_list.append(j)
      if len(filtered_list) != 0: 
        file = open('filteredJobs.txt', 'a', newline='')
        with file:
          write = csv.writer(file)
          write.writerow(filtered_list)

Now we have our list of jobs that we know are developer jobs and they are all in one place.

You can find the code and list on my GitHub

Project code
List of developer jobs

Also if your company is hiring Web Developers check out my portfolio. I am looking to break into tech and am very excited to learn more about your company.

Using The web-voice-command npm Package I Just Published

matoval — Thu, 04 Jun 2020 20:00:50 +0000

npm package: web-voice-command
The Web Speech API is an exciting glimpse into the future of the web. There are a ton of cool ideas I have for using voice to control the web. I will be build some in the next few weeks using the web-voice-command package that I just published.

Goal for this package

My goal is to make it easier to implement the Web Speech API. With this package installed you are able to integrate voice commands and voice-to-text in a few lines of code. Web Speech only works in Chrome at the moment

How to use it

The package has four methods you can call.

setOptions: Used to set options.
results: Returns all voice inputs.
startStop: Used to start and stop listening.
wordList: Returns only words you listed.

setOptions()
Default options are:

  let options = {
    persistentListening: false,
    listenOnLoad: false,
    language: 'en-US'
  }

setting language use BCP 47 Language Codes

ar-SA Arabic Saudi Arabia
cs-CZ Czech Czech Republic
da-DK Danish Denmark
de-DE German Germany
el-GR Modern Greek Greece
en-AU English Australia
en-GB English United Kingdom
en-IE English Ireland
en-US English United States
en-ZA English South Africa
es-ES Spanish Spain
es-MX Spanish Mexico
fi-FI Finnish Finland
fr-CA French Canada
fr-FR French France
he-IL Hebrew Israel
hi-IN Hindi India
hu-HU Hungarian Hungary
id-ID Indonesian Indonesia
it-IT Italian Italy
ja-JP Japanese Japan
ko-KR Korean Republic of Korea
nl-BE Dutch Belgium
nl-NL Dutch Netherlands
no-NO Norwegian Norway
pl-PL Polish Poland
pt-BR Portuguese Brazil
pt-PT Portuguese Portugal
ro-RO Romanian Romania
ru-RU Russian Russian Federation
sk-SK Slovak Slovakia
sv-SE Swedish Sweden
th-TH Thai Thailand
tr-TR Turkish Turkey
zh-CN Chinese China
zh-HK Chinese Hong Kong
zh-TW Chinese Taiwan
Use the setOptions function to change options.

 webVoiceCommand.setOptions({
  persistentListening: true,
  listenOnLoad: true,
  language: 'ro-RO' //set language to ro-RO Romanian Romania
})

results()
The results method is used to get all voice inputs and will return multiple arrays with each array separated by pauses in voice inputs.

 webVoiceCommand.results(function(result){
  console.log(result)
})

With the voice input of "hello world" the results method will return:

(2) ["hello", "world"]

startStop()
The startStop method is used to start and stop listening and is used with a DOM event. In this example we will use two buttons, one for start and another for stop. You can also use the same element as a toggle.

<button value='start' onClick={handleClick}>Start</button>
<button value='stop' onClick={handleClick}>Stop</button>

Now we call the startStop method in the handleClick function. The first parameter is the number of DOM elements and the second is the value "start" or "stop".

function handleClick(event) {
  webVoiceCommand.startStop(2, event.target.value)
}

This will turn listening on/off.

wordList()
The wordlist method is used with a list of words you want the user to use. It can be used with or without actionWords. actionWords are used to trigger lists of words to search through. I will show wordList with and without actionWords.

The wordlist method takes three parameters,

words: Array of objects.
withActions: A boolean.
callback: Returns an array of matched words if you are not using actionWords or a single word if you are using actionWords.

with actionWords

const words = [
  {
    actionWord: {
      name: 'pick',
      options: [
        {
          word: 'blue',
          alternateWord: []
        }, 
        {
          word: 'red',
          alternateWord: []
        }, 
        {
          word: 'yellow',
          alternateWord: []
      }]
    }
  },
  {
    actionWord: {
      name: 'select',
      options: [
        {
          word: 'home',
          alternateWord: []
        },
        {
          word: 'about',
          alternateWord: []
        },
        {
          word: 'contact',
          alternateWord: []
        }
      ]
    }
  }
]

const withActions = true

webVoiceCommand.wordList(words, withActions, function(result) {
console.log(result)
})

With the voice input of "pick blue" the wordList method will return:

blue

without actionWords

const words = [
    {
      word: {
      name: 'red',
      alternateWord: []
      }
    },
    { 
      word: {
        name: 'blue',
        alternateWord: []
      }
    },
    {
      word: {
        name: 'yellow',
        alternateWord: []
      }
    }
  ]

const withActions = false

webVoiceCommand.wordList(words, withActions, function(result) {
console.log(result)
})

With the voice input of "red blue yellow" the wordList method will return:

(3) ["red", "blue", "yellow"]

Conclusion

I will be improving this package as feedback comes in and as I work with it more. My next article will be on a game I am building using the web-voice-command package. All game control inputs will be voice inputs.

Building a Resume For Your First Dev Job

matoval — Tue, 19 May 2020 18:18:21 +0000

Building a resume for your first dev job is difficult because you don't have any employment history to show off on your resume. You might feel like you don't have any experience.

BUT if you have built projects, done any freelance dev work, contributed to open source in any way, then you have meaningful experience that demonstrates your abilities.

Here is how I put my resume together:

I used Figma to build my resume because I am comfortable using it.You can use whichever tool you are most comfortable building in.

Basic Format:
Every resume should show your experience, skills, education, blurb/bio, and contact info.

experience: The code you have written.
Skills: The technologies you know how to use.
Education: What you used to learn your skills.
blurb/bio: A quick summary of who you are as a developer.
contact info: Link to your portfolio site, email, phone number and any other relevant contact info.

The Design
Now we need a nice, simple design that stands out and straight to the point.

Here is my design for my resume:

The Experience Section
The experience section should have a few examples of code you have written, a description of the code, and how it works. You want to convey to the reader of your resume that you know how to use the skills you have. Also remember that most companies have software that screen applicants by using keywords.

Here is my resume with the experience section filled in:

The Skills Section
The skills section should list the skills you have in the development field.

Here is my resume with the skills section filled in:

The Education Section
The education section should list where you learned your skills. If you have a CS degree or graduated from a coding bootcamp you would list that here. But if you are like me, and you are a self-taught developer, then you may list the courses you have completed.

Here is my resume with the education section filled in:

The Blurb/Bio Section
The Blurb/Bio section should be a brief summary of who you are and what you can do as a developer. You want the reader of your resume to be able to understand who you are as a developer even if they only read this section.

The Contact Info Section
The contact info section should have your portfolio site link, email, phone number and any other contact info you would like to add.

Adding Clickable Links
Adding clickable links to your portfolio is a nice touch that allows the reader of your resume a quick and easy way to see a live site of your work.

Resume File Format
You should always export your resume as a PDF and name the file YourName_resume.pdf so that whoever you send your resume to can find it easily.

Here's My Finished Resume

Simple Dark/Light mode

matoval — Wed, 13 May 2020 20:09:29 +0000

Here is the simplest way I found to add dark mode to a project:
Add a toggle button in your HTML with a checkbox and an onClick calling a JavaScript function

 <label id="toggle">
   <input type="checkbox" onclick="toggleDarkMode(event)">
   <span id="slider"></span>
 </label>

Then add a data-mode to your CSS with your CSS variables set to the dark mode colors

:root {
  font-size: 10px;
  --bg: white;
  --font-color: black;
  --btn-bg: black;
  --btn-color: white;
}

[data-mode="dark"] {
  --bg: black;
  --font-color: white;
  --btn-bg: white;
  --btn-color: black;
}

Now all you have left is to write the JavaScript function to set or remove the data-mode

function toggleDarkMode(event) {
  const isDarkmode = event.target.checked

  if (isDarkmode) {
    document.documentElement.setAttribute('data-mode', 'dark')
  } else {
    document.documentElement.setAttribute('data-mode', '')
  }
}

Then you can add some CSS to make the button look better and add a transition

:root {
  font-size: 10px;
  --bg: white;
  --font-color: black;
  --btn-bg: black;
  --btn-color: white;
}

[data-mode="dark"] {
  --bg: black;
  --font-color: white;
  --btn-bg: white;
  --btn-color: black;
}

body {
  width: 100vw;
  font-size: 2rem;
  background-color: var(--bg);
  color: var(--font-color);
}

#toggle {
  margin-right: 2rem;
  width: 6rem;
  height: 3.4rem;
  border-radius: 3rem;
  background-color: var(--btn-bg);
}

#toggle input {
  opacity: 0;
  width: 0;
  height: 0;
}

#slider {
  position: absolute;
  z-index: 5;
  cursor: pointer;
  height: 2.5rem;
  width: 2.5rem;
  top: 1.23rem;
  right: 1.8rem;
  border-radius: 3rem;
  background-color: var(--btn-color);
  transition: ease-in .2s;
}

#slider.dark {
  right: 4.2rem;
}

Here is the end result:

Catching all Responses with Axios

matoval — Wed, 06 May 2020 18:19:18 +0000

We will use a login form to show why adding catch in all Axios calls is so important. When users use a login form there needs to be feedback if they submit the wrong username and/or password. Without a catch in your Axios call, you won't get a response unless the response status is 200. Any other response status you will only get an error in your browser console like the image below:

Without a catch, you can't alert the user that there's an issue and the user doesn't know what the problem is.

Let's take a look at the React code of a login form handleSubmit function.
In this function we are getting the user's inputs and storing them in a user object. Then we send that object in the request body to the server route /users/login.

axios.post('/users/login', user)
      .then(res => {
        props.loggedIn(res.data)
      })
      .catch(error => {
        console.log({
          error, 
          'error status': error.response.status, 
          'error response': error.response.data
        })
        alert('Authentication failed')
      })

Now let's look at the server code that receives this request. We store the email and password that was sent in the body of the request in variables.

Then we search the database for a user using the email we received; this is the first place that might send back an error. If we have a catch in the axios call we can send back json with the error code, and use it to inform the user that 'Authentication failed' by showing an alert. Now the user can check their inputs for errors.

If we find a user in the database we can move forward to checking if the password is correct. We are using bcrypt to hash our password. Bcrypt provides the bcrypt.compare function to check if the password the user submits matches the hashed password in the database. If the password doesn't match, bcrypt.compare will return false and we can again send json with 'Authentication failed'.

With the catch in place we receive the following response in the browser console:

When building a login route you want to return the same response if the username, password or both are wrong. This will prevent giving someone the ability to find usernames in your database and using a word list to hack into your user's accounts.

router.route('/login').post((req, res) => {
  const email = req.body.email
  const password = req.body.password

  User.findOne({email: email})
    .then(user => {
      if(!user) {
        res.status(400).json('Authentication failed')
      } else {
        bcrypt.compare(password, user.hashPW, (err, result) => {
          if (result === true) {
            const refreshToken = makeRefreshToken(user)
            const accessToken = makeAccessToken(user)

            res.setHeader('Cache-Control', 'private')
            res.cookie('refreshToken', refreshToken, { maxAge: 6000000, httpOnly: true})
            res.cookie('accessToken', accessToken, { maxAge:  9000000, httpOnly: true })
            res.status(200).send({userId: user._id})
          } else {
            res.status(400).json('Authentication failed')
          }
        })
      }
    })
    .catch(err => res.status(400).json('Error: ' + err))
})

If the password matches we can return a status of 200 and send the user id back to the React app where we call the loggedIn function with the res.data containing the user id as a parameter.