分类 Miscellaneous 中的文章

Process Pools, Thread Pools, and Coroutines in Python

Neither threads nor processes can be opened indefinitely; they will always consume and occupy resources. Hardware has limited capacity. While ensuring high-efficiency work, hardware resource utilization should also be guaranteed. Therefore, an upper limit needs to be set for hardware to alleviate its pressure, which led to the concept of pools.……

阅读全文

Summary of Python3 print function usage

Python 3 makes the print function more explicit compared to Python 2.

1. Outputting Strings and Numbers

print("runoob") # Outputs string runoob

print(100) # Outputs number 100

str = 'runoob'

print(str) # Outputs variable runoob

L = [1,2,'a'] # List

print(L) [1, 2, 'a']

t = (1,2,'a') # Tuple

print(t) (1, 2, 'a')

d = {'a':1, 'b':2} # Dictionary

print(d) {'a': 1, 'b': 2}

2. Formatted Integer Output

Supports parameter formatting, similar to C language’s printf.

str = "the length of (%s) is %d" %('runoob',len('runoob'))

print(str) # the length of (runoob) is 6

Python String Formatting Symbols:

** Symbol** Description
%c Formats character and its ASCII code
%s Formats string
%d Formats signed decimal integer
%u Formats unsigned decimal integer
%o Formats unsigned octal number
%x Formats unsigned hexadecimal number (lowercase)
%X Formats unsigned hexadecimal number (uppercase)
%f Formats floating-point number, precision can be specified after decimal point
%e Formats floating-point number in scientific notation (lowercase ’e')
%E Same as %e, formats floating-point number in scientific notation (uppercase ‘E’)
%g Shorthand for %f and %e
%G Shorthand for %f and %E
%p Formats variable’s address in hexadecimal

Formatting Operator Auxiliary Directives:

……

阅读全文

Closing a Screen Session from Outside the Session

Screen is an excellent tool for running programs in the background. Typically, you can connect to a session using screen -r and then close the session with Ctrl + C. However, if you’ve implemented exception handling (like try-except) in your Python scripts, you’ll find that this only breaks out of the loop and doesn’t terminate the entire session. In such cases, directly closing a specific session from outside the session is much more convenient.

The command to do this is:

screen -XS test quit

Here, test is the name of your session. Of course, to view all active sessions, you can use:

screen -ls

Here’s an example sequence of commands:

screen -S test python test.py
screen -ls
screen -XS test quit
screen -ls

screen_test

……

阅读全文

Running Conda Command-Line Scripts on Windows

I needed to remotely update website content from my Windows machine at home, and while I have Conda installed on Windows, I found I couldn’t directly write and run scripts. I could only launch Conda from the Start menu and type commands line by line.

After some searching and learning, I finally found a solution.

The reason you can’t directly write and run .bat batch files for Conda commands (which use various Python libraries from Conda) is that the default cmd.exe doesn’t activate the Conda environment. Once you add the environment activation command, your scripts will run. You can refer to the original English explanation here: https://stackoverflow.com/questions/46305569/how-to-make-batch-files-run-in-anaconda-prompt.

The final command structure looks like this:

call 'G:/conda/Scripts/activate.bat' 
scrapy crawl dou
python tupian.py
python post.py
……

阅读全文

Accelerating Python Code with Numba to C++-like Speeds

1. Introduction

Numba is a Just-in-Time (JIT) compiler for Python. This means that when you call a Python function, all or part of your code is converted “on-the-fly” into machine code, which then runs at the speed of your native machine! It is sponsored by Anaconda, Inc. and supported by many other organizations.

With Numba’s help, you can accelerate all computationally intensive Python functions (e.g., loops). It also supports the NumPy library! So, you can use NumPy in your computations and speed up overall calculations, as loops in Python are notoriously slow. You can also use many functions from Python’s standard math library, such as sqrt, etc.


2. Why Choose Numba?

《Accelerating Python Code with Numba to C++-like Speeds》

So, why choose Numba when there are many other compilers like Cython and PyPy? The reason is simple: you don’t have to leave the comfort zone of writing Python code. That’s right, you don’t need to change your code at all to get a speedup comparable to what you’d get from similar type-defined Cython code. Isn’t that great?

……

阅读全文

Python3 Binary Number Representation and Bitwise Operations

In information theory, the Hamming distance between two equally long strings is the number of positions at which the corresponding characters are different. In other words, it’s the minimum number of substitutions required to transform one string into the other. If the strings are binary, the Hamming distance can be calculated very simply using a bitwise XOR operation.

Here, we will introduce how to use binary representation and perform bitwise operations in Python 3.


Binary Representation in Python

In Python, binary numbers can be represented by strings prefixed with "0b" or "-0b". We can also use the bin() function to convert a numerical value into its binary string form.

    print(0b1101)
    #13
    print(0b11111110)
    #254
    bin(5)
    #'0b101'

Binary Bitwise Operations

Python provides the following bitwise operations:

  • >>: Right shift
  • <<: Left shift
  • |: Bitwise OR
  • &: Bitwise AND
  • ^: Bitwise XOR
  • ~: Bitwise NOT

Let’s look at them in detail below.

……

阅读全文

Python implementation for Qianqian Music MP3 download

Entering the Qianqian Music homepage and selecting Jay Chou’s “Confession Balloon” reveals that it’s a 2016 song with no preview available, which is sad. Is there a way to get the MP3 file? The answer is yes. A runnable program for music download is available at the end of the article.

Without further ado, I opened the charts and chose a new song that could be previewed; “Sheng Pi Zi” was the first one I could listen to.

Python implementation for Qianqian Music MP3 download

1. Analyze API information

Opening the developer tools, I found that music files are definitely submitted through an API. Among many requests, I found a request that could retrieve music files. See the image below:

Python implementation for Qianqian Music MP3 download

Check the request details:

Python implementation for Qianqian Music MP3 download The songid parameter can be found in the current URL: http://music.taihe.com/song/611238837. It’s simple. from should be “web” or “app”, etc. The format defining the return data type doesn’t need to be changed. method doesn’t need to be changed. The _ parameter is a 13-digit timestamp. callback is the prefix of the returned JSON data. The 1546915161467 after the underscore is a 13-digit timestamp, while the preceding 17200943498528136486 is unknown. We’ll try to use the known parameters to see if we can retrieve information without changing the unknown content.

……

阅读全文

Python implementation for Qianqian Music MP3 download

Entering the Qianqian Music homepage and selecting Jay Chou’s “Confession Balloon” reveals that it’s a 2016 song with no preview available, which is sad. Is there a way to get the MP3 file? The answer is yes. A runnable program for music download is available at the end of the article.

Without further ado, I opened the charts and chose a new song that could be previewed; “Sheng Pi Zi” was the first one I could listen to.

Python implementation for Qianqian Music MP3 download

1. Analyze API information

Opening the developer tools, I found that music files are definitely submitted through an API. Among many requests, I found a request that could retrieve music files. See the image below:

Python implementation for Qianqian Music MP3 download Check the request details:

Python implementation for Qianqian Music MP3 download

The songid parameter can be found in the current URL: http://music.taihe.com/song/611238837. It’s simple. from should be “web” or “app”, etc. The format defining the return data type doesn’t need to be changed. method doesn’t need to be changed. The _ parameter is a 13-digit timestamp. callback is the prefix of the returned JSON data. The 1546915161467 after the underscore is a 13-digit timestamp, while the preceding 17200943498528136486 is unknown. We’ll try to use the known parameters to see if we can retrieve information without changing the unknown content.

……

阅读全文