Python Tutorial Archives •

How To Manage SQLite Database In Python

Leave a Comment / Python Tutorial / SQLite

SQLite is different from Oracle, MySQL, and other server-level databases. SQLite is just an embedded database engine, which is especially suitable for accessing an appropriate amount of data on devices with limited resources (such as mobile phones, PDAs, etc.).

How To Fix PyCharm Cannot Start Issue

Leave a Comment / Python Tutorial / PyCharm

I use the PyCharm community edition for python development sometimes, but recently when I launch it, it shows a dialog and then exits like a flash. I was confused with this for some time. And now I found how to fix it, I will share the solution in this post.

How To Bypass Anti Web Crawler In Scrapy Project

Leave a Comment / Python Tutorial / Scrapy

The essence of a crawler is to capture valuable data from the target websites. Therefore, every website will more or less adopt some anti crawler technology to prevent crawlers. For example, the user-agent request header is used to verify whether it is a browser, and JavaScript is used to load resources dynamically. These are the …

How To Bypass Anti Web Crawler In Scrapy Project Read More »

Django Create And Export Excel Files With XLWT Example

Leave a Comment / Python Tutorial / Python Django, Python XLWT

This article introduces how to use the Python XLWT library to generate excel spreadsheet tables and export the excel file to HTTP client in Django.

How To Use Python Scrapy To Crawl Javascript Dynamically Loaded Pagination Web Page

Leave a Comment / Python Tutorial / Scrapy

Most websites use the URL link to implement pagination, but some website does not have such pagination links on their web page, they use javascript to load the next page content dynamically when a user scrolls the web page. For example the website https://unsplash.com/. It is an image repository website, it will show next page …

How To Use Python Scrapy To Crawl Javascript Dynamically Loaded Pagination Web Page Read More »

How To Get Redirection URL In Python Crawler

Leave a Comment / Python Tutorial / Python Requests, urllib

When you use Python to write a web crawler, you may encounter the situation that the web page is redirected when you crawl the page. When you request a web page URL, it redirects to another web page URL, the URL address change will result in unable to get the web page content. This article …

How To Get Redirection URL In Python Crawler Read More »

Python Save Html File From Url Example

Leave a Comment / Python Tutorial / Python Requests

This article will tell you how to use the Python requests module to retrieve a web page content by page URL and then save the web page content to a local file step by step.

How To Create Scrapy Project To Crawl Web Page Example

Leave a Comment / Python Tutorial / Scrapy

Scrapy is a Python library that can be used to crawl web pages and extract the web page elements by XPath or CSS selector in python code. This article will tell you how to create a Scrapy project and how to implement the Scrapy related classes in the project to crawl and extract a job …

How To Create Scrapy Project To Crawl Web Page Example Read More »

Python 3 Matplotlib Draw Point/Line Example

Leave a Comment / Python Tutorial / Python Matplotlib

Python matplotlib module is used to draw graphical charts. This article will tell you how to use matplotlib to draw point and line. But before you can use it, you should make sure it is installed. You can open a terminal and input the below command to check, if there is no error message print …

Python 3 Matplotlib Draw Point/Line Example Read More »

How To Fix Scrapy Error Response.xpath Syntax Error Near Unexpected Token

Leave a Comment / Python Tutorial / Scrapy

Scrapy shell is useful to browse a web URL and extract the web element data in an interactive console, but when I use the Scrapy shell to extract a web page item with XPath recently, I meet an error message -bash: syntax error near unexpected token. This article will tell you how to fix it.