5 cool things I am doing with my VPS

As a programmer, I can't imagine the life without a VPS. It plays an important role in my daily life. I want to share 5 cools things I am doing with my VPS.

1. Host my blog

My blog, as you are seeing now, is a static site built by Pelican locally and then rsynced to the VPS from where it's hosted by NGINX. I also generated the wildcard cert for *.yanxurui.cc using acme.sh and renew and then rsync it to other nodes automatically.

2. Host my web app "Snapfile"

Snapfile is a python web app I developped for sharing messages and files quickly. No account is needed. My VPS is also hosting other useful web services like Jupyter notebook which allows me to run any Python scripts (I usually need to do some quick analysis) from any device, including my iPhone. Different http services, including my blog mentioned above are sharing the same public port. This is achieved by NGINX's server_name directive.

3. Proxy via Trojan

As you know, people in mainland china are banned to access a number of necessary services such as Google, Facebook, etc, due to the Chinese internet censorship. Any connection to their servers from China will be dropped by the GFW (Great Fire Wall) immediately. However, that is not big problem for programmers like me though it works for most Chinese people. Trojan is the most effective proxy to workaround the GFW by the time I am writting this artical. Essentially, its protocol can be simplified as a socks5 (works at transport layer) proxy wrapped by TLS. Proxy means we no longer connect to Google directly but connect to the proxy server instead which then communicates with Google on behalf of us. The client can also decide whether to route the traffic to the proxy server according to the hostname or IP, which is super flexible. Of course, this does not work if the Chinese government decides to cut off the submarine cable between China and the rest of the world someday. Similarly, I also have a http proxy (mitmproxy) running on the VPS to intercept and fake the http response using python scripts for some specific urls.

4. Run scheduled tasks

One of my most important cron jobs is to crawl the history prices of a set of stocks from a financial website on each trading day and produces some signals which can help to make investment decisions. The signals are sent to me and few other subscribers via Email. Source code

45 14 * * * cd ~/invest && ~/.pyenv/versions/invest/bin/python monitor.py >> fund.out 2>&1

Why not run this timer on my local computer? Local computer is capable of doing this kind of jobs but it has a big limitation: not always available due to being off or disconnected to the Internet etc. In contrast, a server can be 7x24 hours available.

5. Backup files from my Mac

I have many important files which account for so much storage space in my Mac that I couldn't afford the ICloud storage. I have an automative task to sync my files from my Mac to the VPS for backup purpose. Actually, I also have a second VPS in a different site which is working as a "lag" backup. I bought this concept from microsoft Exchange service. Why 2 backups? Obviously, the secondary backup can be a life-saver if the primary backup is distroyed. The main difference between the 2 backups is that the first backup is synced daily whereas the second one is synced monthly. Why? Suppose I deleted something unintentionally in my Mac and I didn't realize that until a few days later when it was also deleted in the primary backup. In this case, I could still restore it from the lag backup with a very good chance.

On mac, we use launchagents (something like crond on Linux):

create a file /Users/yxr/Library/LaunchAgents/com.yanxurui.rsync.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>Label</key>
        <string>com.yanxurui.rsync</string>
        <key>ProgramArguments</key>
        <array>
                <string>/Users/yxr/rsync_daily.sh</string>
        </array>
        <key>StartCalendarInterval</key>
        <dict>
        <key>Minute</key>
        <integer>00</integer>
        <key>Hour</key>
        <integer>00</integer>
        </dict>
  <key>StandardOutPath</key>
  <string>/tmp/rsync.log</string>
  <key>StandardErrorPath</key>
  <string>/tmp/rsync.log</string>
</dict>
</plist>

rsync_daily.sh

#!/bin/bash
# important: bash must have Full Disk Access on Macos Catalina
# backup files to linux server on a daily basis using launchd (com.yanxurui.rsync)

date
rsync -rlvc -e 'ssh' --delete --exclude {'output','.DS_Store'} ~/Documents ~/Desktop yxr@yanxurui.cc:~/backup/
echo "It took $SECONDS seconds"
echo "------------------------"

# use -a instead when downloading from remote to local
# rsync -av -e 'ssh' hao@47.95.215.105:~/ ./download

Some useful commands

# load before start
cd /Users/yxr/Library/LaunchAgents/
launchctl load -w com.yanxurui.rsync.plist

# start the job immediately (usually for test)
launchctl start com.yanxurui.rsync
# check log file /tmp/rsync.log

# list the job
launchctl list | grep com.yanxurui.rsync

# reload
launchctl unload com.yanxurui.rsync.plist
launchctl load -w com.yanxurui.rsync.plist