English | MP4 | AVC 1280×720 | AAC 44KHz 2ch | 7.5 Hours | 3.50 GB
Become a Professional Web Scraper | Learn how to scrape websites and obtain useful data by using Selenium and NodeJS
Web scraping is the Data Science technique of gathering or storing useful and public information from websites in a suitable format like spreadsheets that can be used later for any purpose. It basically removes the burden of copying the data manually, instead it provides an automated way to do it.
The data extraction is done from websites using some sort of a code written called “scraper” which takes the a Web URL, parses the HTML DOM of that site and then return the result which we can save in some format say CSV for future use. The data that we parse can be in form of texts, images, package names and their corresponding prices, videos and contact information like email, name, phone numbers and so on.
This course is built in NODEJS that is light weight, efficient, and scale-able due to its non-blocking I/O and event driven. The main advantage of node is that its asynchronous in nature which means that you don’t have to wait for a request to complete before making another one which in turn provides a very high performance. This course is made in so much easy way so that one who doesn’t have any knowledge of NODEJS can easily understand and hence build own scraper in NODEJS.
We are using Selenium which is automation testing tool that provides various elements and methods to interact with HTML DOM in order to parse the website.
This course is divided into 3 parts
- Setting up the environment
- Building Blocks and Syntax
- Projects
In the Project Section course will cover 3 Projects
- Google Translate 2018 Project
- Google Translate 2019 Project
- Travel Insurance Website Project
Bonuses:
- Reading Data from CSV File
- How to Save scraped Data into New CSV file
- How to apply Validations
What you’ll learn
- Understands the basics of NODE JS
- Understand the installation and un-installation of node modules
- Understand the fundamentals of Web Scrapping
- Understand the automation tool Selenium with scrapping
- Understand how to locate various elements
- Understand how to locate elements with XPATH
- Understand the basics of Selenium methods that are necessary for scrapping
- Understand different types of Exception in selenium and how to handle them
- Store the scrapped data in excel file CSV
- Complete Project that will Scrape Google translate website
- Complete Project that will Scrape Travel Insurance website
Table of Contents
Whats this Course is about
1 Course Promo
2 Introduction
Setting Up the Environment
3 What is Node JS
4 Installing NodeJS
5 NPM
6 Editor
Basics of Scraping
7 Exercise Files Project Files
8 Scrapping Parts
9 Driver
10 Seleinum Elements
11 Selenium methods
12 Basic Scrape
13 Selenium wait
14 Quitting Browser
15 Writing Method
16 Exception Handling
17 Taking a Live Screen Shot
Project (Google Translate E8)
18 Project Division
19 st Project Preview GT2K18
20 Getting Language id GT2K18
21 Getting Language Related to Array GT2K18
22 Creating Language File GT2K18
23 Deleting Existing File GT2K18
24 Removing Bug and Running the Process in Background GT2K18
25 Building Scrape File GT2K18
26 Scrape Google Translate Function Part 1 GT2K18
27 Scrape Google Translate Function Part 2 GT2K18
28 Adding Data to CSV File GT2K18
29 Limiting the Characters according to Google GT2K18
30 Adding Delays GT2K18
31 Final Testing GT2K18
Project (Google Translate E9)
32 Google Translate E9 Preview GT2K19
33 Opening Language Box GT2K19
34 Getting Class GT2K19
35 Getting Language Id GT2K19
36 Getting Name GT2K19
37 Creating File GT2K19
38 Deleting Existing File GT2K19
39 Reading Client CSV File GT2K19
40 Starting Scrape function GT2K19
41 Running Script GT2K19
42 Exception Handling GT2K19
43 Limiting Translation Content
44 Saving Output Data for Client GT2K19
45 Adding Delay Factor and Testing GT2K19
Project Scraping Travel Insurance Website
46 Project Preview
47 Understanding Test Cases
48 Adding Const and Variables
49 Looping Params
50 Executing Test Case and Sending it to Index.JS file
51 Starting Index File
52 Convert Date Function
53 Selecting 1st Two Fields
54 Trip Duration
55 Finishing Current Page
56 Compiling Results
57 Appending to CSV File
58 Debugging
59 Final Testing Travel Pro
60 Good Bye
61 For More
Resolve the captcha to access the links!