Technology on 春江暮客

Build Your Own TRON Wallet Toolkit (Batch Address Generation / USDT Transfer / Staking & Voting)

Fri, 24 Apr 2026 11:07:45 +0800

Three Python scripts covering the TRON wallet core loop: batch address generation, USDT/TRX transfer, staking/unstaking/voting, and energy delegation.

M3U8 to MP4 Tutorial: Download and Convert M3U8 Videos Using Python or Static HTML

Tue, 27 Jan 2026 13:40:31 +0800

Sometimes you may want to download an M3U8 (HLS) video stream and convert it into an MP4 file for offline viewing or local storage, without relying on complex server-side solutions or paid software.

In this article, I will introduce two practical and commonly used methods to convert M3U8 to MP4:

Using Python with the downloadm3u8 package
Using static HTML with browser-side FFmpeg (FFmpeg.wasm)

Method 1: Convert M3U8 to MP4 Using Python downloadm3u8

This method is suitable for local machines or servers, requires installing dependencies, and is very stable.
It is recommended for large video files or batch downloads.

1. Install Dependencies

First, install ffmpeg (used for video merging) and the downloadm3u8 Python package:

apt install ffmpeg -y
pip install downloadm3u8

2. View Help Information

After installation, check the available options:

downloadm3u8 -h

Example output:

usage: m3u8downloader [-h] [--user-agent USER_AGENT] [--origin ORIGIN]
                      [--version] [--debug] --output OUTPUT
                      [--tempdir TEMPDIR] [--keep]
                      [--concurrency N]
                      URL

3. Download Example

Assume your M3U8 URL is: bobobk test m3u8

Using Matplotlib's fill_between for Area Filling in Python

Wed, 29 Dec 2021 08:47:31 +0800

When plotting with matplotlib in Python, sometimes to enhance the aesthetics of the graph, it’s necessary to fill with color above or below the plot. The ‘fill_between’ function is used for this, where the ‘y2’ parameter determines the second line, indicating whether to fill upwards or downwards.

Foundations of Data Science: Common Probability Distributions and Their Explanations

Thu, 09 Jan 2020 11:11:12 +0800

Probability distributions are as fundamental to statistics as data structures are to computer science. If you want to be a competent data scientist, understanding them is foundational.

How to Convert Between YAML and JSON (Complete Python/JavaScript Guide)

Sat, 17 May 2025 08:53:04 +0800

A comprehensive guide to mastering YAML and JSON conversion using Python and JavaScript, including handling complex structures, cross-language solutions, and performance optimization tips.

How to Improve Website Indexing Speed with IndexNow

Wed, 07 May 2025 10:26:16 +0800

In current SEO practices, rapid indexing of pages by search engines is a key concern for webmasters. Traditional search engines rely on crawlers to periodically fetch website content, resulting in low indexing efficiency. IndexNow is a proactive submission protocol that allows websites to notify search engines of content updates in real-time, significantly improving indexing speed. This article will explain how to use IndexNow for efficient indexing.

Automatically Publishing Articles to WordPress Using a Python Script: A Complete Workflow Analysis

Mon, 21 Apr 2025 08:25:00 +0800

When building content sites or AI content distribution projects, how can you efficiently batch publish generated content to WordPress? This article will share how I used a Python script to build an automated content publishing process that does not rely on the WordPress backend, does not require XML import, and directly connects to the database for maximum efficiency.

Python: Creating Beautiful Lollipop Charts

Fri, 31 May 2024 10:03:32 +0800

The lollipop chart, as a variant of bar charts and dot plots, is increasingly popular in scientific research due to its simplicity and clarity. This chart cleverly combines the advantages of bar charts and scatter plots, offering a visually more attractive way to represent data.

Downloading m3u8 Streaming Media Using Python and Embedding into AMP Webpages

Tue, 01 Aug 2023 09:02:57 +0800

In the era of short videos and live streaming, video formats are often no longer traditional ones like mp4 or mkv, but streaming formats such as m3u8. If you want to download your favorite videos, special handling is needed. This article provides a detailed example of how to download m3u8 streaming data using Python and embed it into an AMP webpage for playback. Let’s dive into the full process.

Managing Long-Running Tasks on Unix-like Systems

Fri, 21 Jul 2023 14:47:07 +0800

Due to daily work all using Linux systems, generally for commands known to run for a long time, nohup, screen, or tmux are used to run the command. However, sometimes after starting a command, you suddenly realize it runs very long. Is there a way to continue working in the current shell and keep the command running after logout until the task finishes?

ChatGPT Automatically Generates Web Pages

Thu, 16 Feb 2023 16:57:33 +0800

This time, I used the viral ChatGPT to generate a web page that counts words. I asked it to create a web page to count the number of words in a text area using HTML, CSS, and JavaScript.

Using Postfix, Dovecot, MySQL, and roundcube to build a mail server under Debian

Sat, 18 Jun 2022 19:41:47 +0800

There are many articles on the Internet about using Postfix to build a mail server, but none of them can be built successfully. After many destructions and reconstructions, the mail system was finally completed and can send and receive mails smoothly. This article will introduce how to use Postfix+Dovecot+MySQL+roundcube to build a mail server in Debian system

Using SomaticSignatures to Identify Mutation Signatures from Mutation Data

Sat, 09 Apr 2022 11:41:37 +0800

The SomaticSignatures package was published in 2015 in the bioinformatics journal, a professional bioinformatics journal. This package aims to analyze tumor single-nucleotide variants (SNP) data to discover tumor development and evolution mechanisms. This article introduces how to analyze snv data to obtain tumor-specific SNP signatures.

Designing Baidu SEO Click Software with GUI using PyQt6

Wed, 30 Mar 2022 08:39:43 +0800

For webmasters, improving search engine rankings through website SEO and attracting more organic users is crucial for continuously increasing website traffic. In addition to improving the quality of their own website content, simulating searches and clicks on search engines is a complementary method. After comparing various existing Baidu and Sogou click software, designing a customizable SEO click software of one’s own seems very challenging and meaningful. Recently, I’ve also been learning PyQt6 as a GUI programming package, which is perfect for this task.

How to Install Old Version R Packages in R

Tue, 15 Mar 2022 09:01:38 +0800

To leverage the efficiency improvements and bug fixes of R version updates, I upgraded R on my server to the latest version (currently 4.1). However, when using some packages, I encountered errors.

Drawing Raincloud Plots with Python

Tue, 11 Jan 2022 15:56:31 +0800

When performing exploratory analysis, bar charts and box plots are excellent methods that can effectively display the general data structure and distribution. Recently, I saw someone using raincloud plots to display data, and the graphics looked beautiful and interesting. Therefore, I have organized this information and implemented the drawing of raincloud plots using Python.

Introduction to Canonical Correlation Analysis and Python Implementation

Wed, 29 Dec 2021 14:01:33 +0800

When handling high-dimensional data, we can use LDA, PCA, etc., for dimensionality reduction. But what if two datasets come from the same samples but differ in data types and scales? This is where Canonical Correlation Analysis (CCA) becomes useful.

Calculating Confidence Intervals Using Bootstrapping

Tue, 28 Dec 2021 10:19:45 +0800

Confidence interval (CI) is the range within which the population parameter lies with a certain confidence level. It is estimated based on the original observed sample and is usually defined as 95%, commonly referred to as the 95% confidence interval.

Choice of Preprocessing Methods: Scale, Standardize, Normalize

Sun, 26 Dec 2021 15:43:24 +0800

Many machine learning methods require data to be approximately normally distributed and as close as possible to normality. In Python, sklearn is the popular package for machine learning, providing functions including MinMaxScaler, RobustScaler, StandardScaler, and Normalizer for preprocessing.

The c-index and Its Application in Survival Analysis

Thu, 23 Dec 2021 17:06:46 +0800

The concordance index (c-index) is a metric used to evaluate the performance of predictive models, particularly in survival analysis. It is defined as the proportion of concordant pairs at all time points.

Python Native Lists vs. NumPy Arrays

Wed, 22 Dec 2021 09:49:00 +0800

In Python, you can choose from various native data types to store collection data, including list, array, tuple, and dictionary. Among these, the list is highly flexible, can store any content, and is mutable, making it widely applicable. However, for scientific computing and storing purely numerical data, NumPy is widely used and has practically replaced lists. So, what are the differences between them, how significant are these differences, and how should they be applied in practice?

K-Means Clustering and Implementation with sklearn

Mon, 20 Dec 2021 08:19:34 +0800

Cluster analysis allows us to find groups of similar samples or features, with stronger correlations among these objects. Common uses include grouping samples by different gene expression profiles or grouping genes by classifications of different samples.

Four Plotting Tips in Python

Mon, 13 Dec 2021 18:25:17 +0800

When visualizing data, it’s common to plot multiple charts in a single figure. For example, visualizing the same variable from different perspectives like side-by-side histograms and boxplots for numerical variables is useful.

Using Docker to Render JavaScript Web Pages via Browser from Command Line

Wed, 08 Dec 2021 08:58:40 +0800

When using scrapy to crawl web pages, many websites render content with JavaScript, so directly fetching the source code will not get the needed content. In this case, using selenium to drive a browser to get the rendered content is very suitable.

Accelerating pip and anaconda in China

Mon, 06 Dec 2021 09:23:35 +0800

Since the default addresses for pip and anaconda are very slow to access in China, adding domestic mirrors for acceleration is necessary.

Hands-on Implementation of Random Forest Algorithm with Python

Mon, 28 Dec 2020 14:04:47 +0800

This article will guide you through a hands-on implementation of a powerful random forest machine learning model. It aims to complement my conceptual explanation of random forests, but as long as you have a basic understanding of decision trees and random forests, you can fully read it. Later, we will discuss how to improve the model built here.

Summary of various string segmentation methods in Python

Mon, 09 Nov 2020 10:46:15 +0800

Today I will introduce to you various string segmentation methods that can be used in Python. They are

Python Script to Snatch Recently Expired Domains

Tue, 22 Sep 2020 12:02:10 +0800

‘Many domain enthusiasts scour forums and websites frantically searching for and snatching up suitable domains, even spending heavily to buy desired domains from their owners. International domain management bodies adopt a “first-to-apply, first-to-register, first-to-use” policy. Since domains only require a small annual registration fee, continuous registration grants you the right to use the domain. Because of this, many domain resellers (commonly known as “domaining pros”) often spend heavily on short, easy-to-remember domains. I used to think about buying shorter domains for building scraping sites, but unfortunately, both snatching and buying from others were very expensive. Since it"s first-come, first-served, we can also acquire good domains by registering them before the current owner forgets to renew.’

python3 solution to LeeCode medium problem

Wed, 29 Jul 2020 09:54:36 +0800

This is an article analyzing a problem from the coding practice site LeeCode.

How to Properly Make Venn Diagrams Using Python Packages

Wed, 08 Jul 2020 10:06:56 +0800

In Venn diagrams of two sets, there can be two (or more) overlapping circles representing sets of different sizes, but the circles are the same size. Actually, the circles should be proportional to the size of the sets, and the overlapping area should also be proportional to the data overlap.

Calculating the Gini Coefficient and Plotting the Lorenz Curve with matplotlib

Sun, 31 May 2020 19:22:50 +0800

The Gini coefficient and Lorenz curve are widely used to represent data inequality, especially wealth inequality. However, currently in Python, there isn’t a very good function to directly plot the Lorenz curve. Since the current project requires it, this article records how to use numpy, pandas, matplotlib, and other packages to calculate the Gini coefficient and plot the Lorenz curve for practical use.

Bayesian Theory and Practical Python Applications

Sun, 31 May 2020 11:58:57 +0800

Bayesian theory provides a principled method for calculating conditional probabilities. With it, we can easily compute conditional probabilities for events where intuition often fails.

Fetching Stock Data Using Python's yfinance Package

Sat, 09 May 2020 08:54:26 +0800

Historical stock data is a very important kind of time series data, playing a significant role in data science. Let’s start learning how to handle time series data, preparing for future stock prediction and analysis.

Using folium to Draw a COVID-19 Pandemic Map

Sun, 03 May 2020 08:28:47 +0800

After being contained in China, the COVID-19 pandemic became increasingly severe worldwide. Countries and regions publish daily new infection and death data to help fight the pandemic globally.

Mounting Oracle Object Storage using s3fs-fuse

Fri, 01 May 2020 08:51:28 +0800

My previous article, “Getting Free Oracle Cloud Servers and Automating Deployment with Scripts,” explained how to use the CLI to acquire free Oracle machines. Oracle’s free tier offers a total of 100GB of disk space, but you can also mount an additional 20GB of object storage as a local file system. This is especially convenient for data migration, as the data can be easily mounted to another instance. This article will show you how to enable free Oracle Object Storage and mount it as a local drive on a Linux system.

Efficient Ways to Check If a List or Tuple Is Empty in Python

Tue, 28 Apr 2020 07:26:37 +0800

In Python, to check whether an array or tuple is empty, there are three methods: comparing with an empty list, checking the length, and using an if statement.

Deploying a Flask Docker App Using Okteto’s Free Container

Mon, 27 Apr 2020 10:04:37 +0800

I just saw a post on the forum about Okteto’s free containers, and since I hadn’t registered before, I decided to deploy my site’s Flask app using Okteto’s free container. Okteto’s free Kubernetes plan offers generous resources: 4-core CPU, 8GB RAM, 10GB SSD — basically a great VPS for free. Here’s how to deploy using Okteto.

Detailed Explanation of Confusion Matrix in Machine Learning

Sun, 26 Apr 2020 20:30:36 +0800

In machine learning, after collecting, cleaning data, and designing preprocessing algorithms, how do we know the effectiveness of the algorithm? How to evaluate if the predictive model can classify effectively and how accurate the classification is? This involves the confusion matrix, which is widely used to evaluate classification problems in machine learning.

Compiling Custom LEDE Firmware for K2 Router Using GitHub Actions

Mon, 16 Mar 2020 13:37:38 +0800

I’ve been using LEDE OpenWRT firmware as my router’s firmware. Due to version updates, some software is missing in older versions, so I decided to compile a custom firmware containing only the needed software for my Phicomm K2 router.

Use Python to set image background color to transparent

Sun, 12 Jan 2020 16:10:01 +0800

Since the original logo has a white background, and we need to change the white background to transparent, here we use the versatile Python tool for image conversion, specifically the pillow package’s Image.

Application of Python Implementation of Gradient Descent in Practice

Wed, 08 Jan 2020 14:04:06 +0800

Gradient descent is a first-order optimization algorithm, commonly called the steepest descent method. To find a local minimum of a function using gradient descent, one must iteratively move from the current point in the opposite direction of the gradient (or approximate gradient) by a specified step size.

Mining Monero on MinerGate Pool Using CPUMiner-Multi on CentOS7

Tue, 07 Jan 2020 11:22:10 +0800

Too many idle servers collecting dust. Here is an option to make use of idle VPS. Today I will teach you how to use CPUMiner-Multi on CentOS for mining and earn some cryptocurrency. Monero is currently the most profitable cryptocurrency, so we choose Monero.

git push update rejected because the remote repository contains commits not present locally

Mon, 06 Jan 2020 15:58:15 +0800

After improving a Hugo theme used by this blog, an error occurred when pushing to the git repository.

Decision Trees in Machine Learning and Python Examples

Sat, 04 Jan 2020 10:49:02 +0800

A tree in real life has many branches and leaves. In fact, the concept of trees is widely used in machine learning, covering classification and regression. In decision analysis, decision trees are used to visually represent and make decisions. As the name implies, a decision tree is a tree-shaped decision model.

The Mathematical Principles Behind PCA and Python Example Demonstration

Fri, 03 Jan 2020 14:35:46 +0800

This article introduces a very important dimensionality reduction method in machine learning—Principal Component Analysis.

Explanation of bit, byte, Bps, bps, kbps, Bps, Gbps, pps and other bit/byte network speed units

Thu, 02 Jan 2020 15:01:36 +0800

It is always known that bit and byte are two concepts, one is binary digit, and the other is a byte unit consisting of 8 bits. However, in practical situations, people still confuse bit and byte in various network speed abbreviations. This article summarizes the knowledge I found.

Statistical Skew Distributions Reveal Statistical Traps in Life

Fri, 27 Dec 2019 16:06:04 +0800

90% of drivers believe their driving skills are above average, 90% think their IQ is above the average IQ of the population, and the key is this can actually be consistent with real data — it’s true and not fabricated.

Command-line UI tool for monitoring computer CPU: s-tui

Mon, 23 Dec 2019 14:26:01 +0800

s-tui is a free and open-source command-line UI tool for monitoring computer CPU. It can monitor CPU temperature, frequency, etc. in real-time and display it graphically. This is a Python tool that can be installed via pip, but requires root permissions. It can be used via SSH login to monitor CPU status graphically.

Installing ffmpeg on CentOS 7 and Streaming Movies on Douyu

Mon, 09 Dec 2019 17:39:34 +0800

Getting back on track, today’s tutorial by Chunjiang Muker shows how to automatically stream Douyu live on CentOS 7 to achieve 24/7 uninterrupted streaming.

How to Use Tongji University Public Mirror for CentOS

Sun, 08 Dec 2019 20:21:22 +0800

A few days ago on WeChat, I discovered that Tongji University actually has its own open source software mirror site. Our lab servers are still using Alibaba Cloud’s CentOS 7 mirror. Wouldn’t it be much faster if we switched to the university’s mirror? So how to change CentOS 7 public repo to Tongji University’s mirror?

Predicting Virmach Black Friday VPS Prices Using sklearn Linear Regression and Alerts

Wed, 04 Dec 2019 20:14:26 +0800

Virmach’s Black Friday machines appear and disappear rapidly. For hosting enthusiasts, constantly checking current VPS prices and deciding whether to buy is troublesome. This article lists Black Friday machine configurations using sklearn and predicts VPS prices with machine learning to alert users if prices are cheap.

VPS Performance and Network One-Click Test Script

Mon, 02 Dec 2019 09:04:01 +0800

During the recent Black Friday sales, I grabbed a special machine from Virmach, so I thought of testing its performance by running some VPS performance and network tests since the machine looks pretty good.

Using find and sed to Batch Replace Strings in Text

Sun, 24 Nov 2019 16:11:25 +0000

How to batch replace strings in Linux? At first, I thought about using sed with -r or some iterative method, but found sed doesn’t support iteration parameters. So, I use find to get files and sed to replace strings.

Manually Create Custom System Service on CentOS7

Sun, 17 Nov 2019 16:11:25 +0000

During Linux system development, sometimes your program might crash or be stopped. To keep it running continuously, adding the program to the service list is a very good practice.

Solving Expert-Level Sudoku Puzzles Quickly Using Python's Backtracking Algorithm

Mon, 11 Nov 2019 16:11:25 +0000

I often play Sudoku in my leisure time as a form of relaxation. My usual method involves eliminating duplicates and filling in unique numbers first, then proceeding step by step. However, it’s inevitable to guess numbers and adjust based on feedback. So, is there a better algorithm to solve Sudoku puzzles? Here, I will use the backtracking method in Python to solve 9x9 expert-level Sudoku puzzles.

Reducing /home partition size and increasing /root space in CentOS 8

Sat, 02 Nov 2019 16:11:25 +0000

I recently set up a CentOS 8 virtual machine to experience the latest CentOS system, allocating 127GB of space. Due to actual needs, I found that the /home partition had tens of gigabytes of space.

Configuring IPv6 Passthrough for Padavan Router to Enable IPv6 for All Internal Hosts

Fri, 01 Nov 2019 00:00:00 +0000

Both China Telecom and CERNET support IPv6, but router configurations often go wrong. This article explains how to set up a K2 router with Padavan firmware to ensure all hosts under the router have IPv6.

Using requests and multiprocessing for multi-threaded brute-force cracking of default lnmp mysql password

Wed, 23 Oct 2019 00:00:00 +0000

My blog used the lnmp program to build the web environment, but when I needed to use mysql, I found that the lnmp installation log file was missing and the mysql root password was forgotten. I remember that the lnmp was installed by pressing Enter all the way, and the default lnmp password is lnmp.org# plus 5 random numbers, with only 100,000 possibilities. In this case, brute-forcing 100,000 times will definitely find the password.

Deploying an SSH Honeypot with Docker to Record SSH Login Passwords

Mon, 21 Oct 2019 16:11:25 +0000

Today I logged into the server and found records of SSH brute force attacks, so I thought I might as well record the hacker’s passwords and try logging into his server in reverse.

10 Tips to Improve Your Python Data Analysis Skills

Mon, 08 Jul 2019 04:34:47 +0000

In programming, even small tips or tools can make a big difference.
For example, a shortcut key or a helpful package might simplify a lot of work and double your efficiency.
Here I’ll share a few small tricks I often use.

1. Use `pandas_profiling` to Inspect DataFrames

Understanding your data is essential before doing any analysis.
Although df.describe() and df.info() provide basic summaries, they’re limited with large or complex datasets.
The pandas_profiling library offers detailed profiling through profile_report().

Installation

pip install pandas-profiling
# or
conda install -c anaconda pandas-profiling

Usage

It’s very easy to use:

import pandas as pd
import pandas_profiling

df = pd.read_csv("train.csv")
df.profile_report()

You can also export the report to HTML:

html = df.profile_report(title='Titanic Profiling Report')
html.to_file(outputfile="titanic_Profiling_Report.html")

2. Interactive Plotting with `cufflinks`

Pandas has built-in plotting via .plot(), but it’s not interactive.
If you want interactivity, try the cufflinks package.

Serialization and Deserialization in Python

Mon, 01 Jul 2019 10:36:01 +0000

Sometimes you may need to temporarily store data so that it can be called directly the next time the program runs, or exchanged between different threads. Serialization is a way to store data in this way, and here we explain Python’s serialization and deserialization using the pickle package.

Python Data Visualization - The Post-2000 Gaokao Generation

Sat, 01 Jun 2019 17:50:52 +0000

The post-2000 generation has finished their Gaokao (National College Entrance Examination), and there’s been extensive media coverage (they are the “fresh meat” generation, after all!). Many reports focused on this year’s examinee data, presenting it with stunning charts. Feeling a bit jealous about how beautiful those charts are? Do you want to try making one yourself? These charts are actually products of data visualization created with Python, so yes, you can definitely make them yourself!

Preparation

Libraries
- charts
- pyecharts
Data
- Collected directly from Baidu.

Common Chart Types

Bar charts and line charts are frequently seen and used, so let’s start with the basics.

1. Bar Chart

# Number of Gaokao examinees
gaokao_num = [940,940,...,375]
gaokao_num.reverse()
# Number of admitted students
luqu_num = [700,705,...,221]
luqu_num.reverse()
# Admission rate
luqu_lev= [74.46,75,...,59]
luqu_lev.reverse()

import charts

options = {
    'chart'   : {'zoomType':'xy'},
    # Title
    'title'   : {'text': '2000-2017 Gaokao Data'},
    # Subtitle
    'subtitle': {'text': 'Source: edu.sina.com.cn'},
    # X-axis
    'xAxis'   : {'categories': ['2000',...,'2017']},
    # Y-axis
    'yAxis'   : {'title': {'text': 'Million people/year'}},
    }
series =  [{
    'type': 'column',
    'name': 'Number of Gaokao Examinees',
    'data': gaokao_num
},{
    'type': 'column',
    'name': 'Number of Admitted Students',
    'data': luqu_num
}
]
charts.plot(series, options=options, show='inline')

Due to a minor issue with my pyecharts setup, I used the charts library. Using pyecharts is even simpler, but I won’t repeat it here. You can check the source code if needed.

Parallelism in One Line of Python Code

Sat, 01 Jun 2019 17:47:59 +0000

Python has a somewhat notorious reputation when it comes to program parallelization. Technical issues aside, such as thread implementation and the GIL, I believe incorrect teaching guidance is the main problem. Common classic Python multithreading and multiprocessing tutorials often seem “heavy” and tend to scratch the surface without deeply exploring the most useful content for daily work.

Longest Palindromic Substring Algorithm - Manacher

Thu, 28 Mar 2019 07:17:29 +0000

While solving LeetCode problems, I encountered a question about finding the longest palindromic substring.

Finding Common Values in Two Python Lists

Fri, 08 Mar 2019 08:23:32 +0000

In daily life, we often encounter the need to find common values between two arrays. This article provides several simple and practical methods on how to elegantly get common values between two arrays in Python.

Python Implementation of Classic Sorting Algorithms (1)

Tue, 29 Jan 2019 07:29:52 +0000

In computer science, a sorting algorithm is an algorithm that arranges a list of data in a specific order. The most commonly used sorting methods are numerical order and lexicographical (dictionary) order. Efficient sorting algorithms are crucial in various other algorithms. Sorting algorithms are also used in processing text data and generating human-readable output.

Basically, the output of a sorting algorithm must adhere to the following two principles:

The output result is an increasing sequence (increasing refers to the desired sort order).
The output result is a permutation or rearrangement of the original input.

The 10 classic sorting algorithms can be divided into two main categories:

Non-linear time comparison-based sorting: These algorithms determine the relative order of elements by comparing them. Since their time complexity cannot break through $O(n log n)$, they are called non-linear time comparison-based sorting algorithms.

Linear time non-comparison-based sorting: These algorithms do not determine the relative order of elements by comparison. They can break through the lower bound of comparison-based sorting and run in linear time, hence they are called linear time non-comparison-based sorting algorithms.

Faceswap Training Resource Acquisition and Processing

Wed, 23 Jan 2019 14:19:38 +0000

Following the previous article, “Faceswap Deep Learning AI for Video Face Swapping Explained,” everyone has learned how to perform face replacement in videos using AI through model training.

Detailed Examples of Seaborn Plotting Kernel Density Curves

Sun, 20 Jan 2019 11:41:58 +0000

In a frequency distribution histogram, when the sample size is sufficiently enlarged to its limit, and the bin width is infinitely shortened, the step-like broken line in the frequency histogram will evolve into a smooth curve. This curve is called the density distribution curve of the population.

In this article, Chunjing Muke will detail how to use the Python plotting library Seaborn and the Iris flower dataset from Pandas to plot various cool density curves.

1. Basic Density Curve

    import seaborn as sns
    sns.set(color_codes=True)
    sns.set_style("white")
    df = pd.read_csv('iris.csv')
    sns.kdeplot(df['sepal_width'])

To plot a kernel density curve using Seaborn, you only need to use kdeplot. Note that a density curve only requires one variable; here we choose the sepal_width column.

2. Density Curve with Shading

    import seaborn as sns
    sns.set(color_codes=True)
    sns.set_style("white")
    df = pd.read_csv('iris.csv')
    sns.kdeplot(df['sepal_width'],shade=True)

Detailed Explanation of Faceswap Deep Learning AI for Video Face Swapping

Fri, 18 Jan 2019 09:29:15 +0000

Chunjing Muke will introduce the super popular black technology application deepfake, an app that achieves face swapping in images and videos.

Drawing a Stunning "Dream of the Red Chamber" Word Cloud with Python 3

Thu, 17 Jan 2019 03:21:06 +0000

Word clouds, which I’m sure you’ve all seen, are created using wordcloud, a famous Python library. This article will detail how to use wordcloud to create a word cloud for “Dream of the Red Chamber,” one of China’s Four Great Classical Novels.

1. Preparation

This involves three parts:

2. The wordcloud and jieba libraries, which can be installed using `pip install wordcloud` and `pip install jieba`.

3. Preparing a Chinese font file.

The `.txt` text file and font file are bundled together for your convenience to replicate this tutorial’s example.

2. Drawing the “Dream of the Red Chamber” Word Cloud

Here’s the code directly:

    from wordcloud import WordCloud
    import jieba
    text = "".join(jieba.cut(open("红楼梦.txt").read()))
    wordcloud = WordCloud(font_path="kaibold.ttf").generate(text)

    # Display the generated image:
    plt.imshow(wordcloud, interpolation='bilinear')
    plt.axis("off")
    plt.margins(x=0, y=0)
    plt.show()

TypeError: ufunc 'isnan' not supported for the input types - Solution

Mon, 14 Jan 2019 12:42:00 +0000

Today, while using Python’s Seaborn to plot a heatmap (clustermap), I kept encountering this error. My data seemed perfectly fine, and a Google search didn’t yield any good solutions. After some exploration, I’m sharing the final solution here.

1. Generating the DataFrame

    import pandas as pd
    import numpy as np
    import matplotlib.pyplot as plt
    from seaborn import clustermap
    import seaborn as sns; sns.set(color_codes=True)
    df = pd.DataFrame([["a","b","c","d","e","f"],[1,2,3,4,5,6],[2,3,4,5,6,7],[3,4,5,6,7,8]],  columns=list('ABCDEF')).T
    df
    g = sns.clustermap(df.iloc[:,1:],cmap="PiYG")

After generating and transposing the DataFrame, a TypeError occurs: TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule "safe".

2. Cause of the Error

This type of error arises because the DataFrame has been transposed, and the original DataFrame contained string columns. Just like in the example above, the first column contains strings (values ‘abcdef’). When transposed, all numerical values in the DataFrame are also converted to object types instead of float or int numerical types. Therefore, trying to plot a heatmap with character types naturally leads to an error.

TypeError: ufunc 'isnan' not supported for the input types - Solution

Mon, 14 Jan 2019 12:42:00 +0000

After generating and transposing the DataFrame, a TypeError occurred: TypeError: ufunc ‘isnan’ not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule “safe”.

2. Cause of the Error

This type of error occurs because the DataFrame has been transposed, and the original DataFrame contains a column with strings. Just like in the example above, the first column contains string values “abcdef”. After transposition, all numbers in the DataFrame also become “object” type instead of “float” or “int” numeric types. Therefore, when we try to plot a heatmap with character types, an error naturally occurs.

If the DataFrame originally contained only numeric types, there would be no issue here.

3. Solution

Knowing the cause, the solution is simple: convert the corresponding numeric columns in the transposed DataFrame to numeric types. Here’s the code:

Python Implementation for Kugou Music MP3 Download

Sun, 13 Jan 2019 14:45:59 +0000

After implementing python for Qianqian Music mp3 download, some users found that many songs couldn’t be searched on Qianqian Music. So today, Chunjian Muke extended the download functionality to Kugou Music, with source code provided.

Using the same approach, first search for a song directly on the Kugou official website. Then, open the network monitor in Google Chrome and search for the same keyword again. You’ll then be able to find the API information (Note: It’s best to view the network requests during the second search to filter out unnecessary information).

1. Analyzing Search API Information

With only 4 network requests, it’s easy to identify that the first request genuinely returns song information, so we can construct this request.

Drawing NetworkX Network Graphs in python3

Sat, 12 Jan 2019 01:58:34 +0000

NetworkX is a Python library for studying graphs and networks. NetworkX is free software released under the BSD-new license. It can be used to create and manipulate complex networks, and to study the structure and function of complex networks.

With NetworkX, you can load or store networks in standard or non-standard data formats. It can generate many types of random or classic networks, analyze network structure, build network models, design new network algorithms, and draw networks.

Of course, NetworkX alone cannot be powerful. Here, Chunjian Muke will use other widely used common Python libraries to draw various basic network graphs.

1. Drawing the Most Basic Network Graph

A network graph consists of nodes and edges. In NetworkX, each row of a pandas DataFrame represents the points in a connection, and a connection is generated at the corresponding position. In the example, a connection is generated between each corresponding position of ‘from’ and ’to’.

Drawing Violin Plots with Seaborn

Fri, 04 Jan 2019 12:02:11 +0000

Introduction

A violin plot is used to display the distribution and probability density of multiple data groups. Similar to a box plot, it offers a better representation of data density. Violin plots are particularly useful when dealing with very large datasets that are difficult to display individually. Python’s Seaborn package makes it very convenient to create violin plots.

Parameters

The parameters corresponding to each position in a violin plot are shown above. The middle line represents the box plot data, specifically the 25th, 50th (median), and 75th percentiles. The thin lines indicate the 95% confidence interval.

Drawing Violin Plots with Seaborn

Single Variable Data

While a box plot would suffice for a single variable, a violin plot can certainly be used as well:

    import seaborn as sns
    sns.set(color_codes=True)
    sns.set_style("white")
    df = sns.load_dataset('iris')
    sns.violinplot( y=df["sepal_length"] )

The difference between shadowcopy and deepcopy in python

Sat, 29 Dec 2018 02:22:22 +0000

In python, it is common to need to copy specific objects, and you might encounter various bugs because understanding the difference between these three operations is key: assignment, shallow copy, and deep copy.

Assignment (=), shallow copy (copy), and deep copy (deepcopy) are relatively easy to distinguish regarding assignment vs. copying, but shallow copy and deep copy are harder to differentiate.

The assignment statement does not copy the object; it simply binds the variable to the object. Any change to one object will affect the other. Copying allows you to change one object without affecting the other.

The difference between shallow and deep copy is that shallow copy does not affect the other object when values change, but adding or deleting elements can affect it. Deep copy creates a completely independent object, and changes to one will not affect the other.

python3 requests module usage examples

Fri, 28 Dec 2018 04:57:52 +0000

The network module in python3 is much more convenient compared to python2. The requests package combines several python2 packages. This article explains the usage of requests with examples, serving as a review and future reference.

Kaggle Local Dog Breed Recognition

Sat, 15 Dec 2018 10:36:29 +0000

After setting up the deep learning environment, the first thing is to try if it works. The simplest way is to use a pretrained model to predict new samples.

Introduction to Artificial Neural Networks

Mon, 10 Dec 2018 05:27:33 +0000

Artificial Neural Network (ANN), also called Neural Network (NN) or neural-like network, is a mathematical model that mimics the structure and function of biological neural networks. It consists of a large number of neurons connected for computation. In most cases, artificial neural networks can change their internal structure based on external information, making them adaptive systems, simply put, they have learning capabilities.

Recursively download files python

Sat, 08 Dec 2018 06:42:32 +0000

I want to back up my website recently, but the size of the file downloaded by PHP is limited, and I am too lazy to install FTP to download it. So I thought of temporarily setting up a secondary domain name site, and then using python (python3)’s requests library to directly download all the files and folders in the root directory of the website to achieve the purpose of backup.

Python Converts All Site Images to WebP Format

Tue, 04 Dec 2018 15:32:34 +0000

WebP is an image file format developed by Google that provides both lossy and lossless compression. The WebP image format can significantly reduce file size while achieving the same image quality as the JPEG format, thereby significantly reducing the time it takes for image files to be sent over the network.

scrapy crawl subtitle group today's updated movies - with source code

Wed, 21 Nov 2018 04:23:53 +0000

Because I like watching American TV shows, and the subtitle group’s resources update very quickly, and I have written many scrapy spiders, and I also maintain a small movie site for friends to download, I wondered if I could use a script to realize daily updates and crawling of subtitle group movie resources.

Drawing a Circle with Squares Using Turtle

Mon, 19 Nov 2018 04:38:15 +0000

I recently discovered a very interesting Python drawing library called Turtle. Here, I’ll briefly explain the idea behind using the Turtle library to draw a circle with squares.

Drawing the Butterfly Curve with Python

Wed, 07 Nov 2018 08:03:32 +0000

The butterfly curve, discovered by Temple H. Fay, is a beautiful curve that can be expressed using a polar coordinate function. Because of its elegance, I wanted to use it as my blog’s favicon.ico. Here, I’ll use Python’s matplotlib.pyplot package to draw the desired butterfly curve. First, let’s admire the beautiful butterfly curve.

1. First, We Need to Define the Mathematical Expression of the Butterfly Curve

It can also be expressed using spherical coordinates:

2. Choosing `matplotlib.pyplot` as the Plotting Tool in Python

1. First, import the necessary Python packages

import numpy as np
import matplotlib.pyplot as plt

2. Set the parameter values

t = np.arange(0.0, 12*np.pi, 0.01)
x = np.sin(t)*(np.e**np.cos(t) - 2*np.cos(4*t)-np.sin(t/12)**5)
y = np.cos(t)*(np.e**np.cos(t) - 2*np.cos(4*t)-np.sin(t/12)**5)

3. According to the formula, use `numpy` functions with `plt` to draw the required image

plt.figure(figsize=(8,6))
plt.axis('off')
plt.plot(x,y,color='blue',linewidth = '2')
#plt.show()
plt.savefig("butter.jpg",dpi=400)

4. Use `Pillow` to resize the image to an appropriate size for a favicon

from PIL import Image
im = Image.open("butter.jpg")
favicon = im.resize((50,50))
favicon.save("favicon.ico")