Morris on 300.Watts

Async in Traits Just Save Us

morristai01@gmail.com (Morris) — Sun, 10 Mar 2024 10:04:55 +0800

The Challenge of Implementing the Future Trait for Custom Types

Developers who venture into crafting their own async/await implementations in Rust may encounter the intricate task of implementing the Future trait for their custom types. Rust’s approach to async/await is nuanced, offering a stark contrast to languages like Go, which employ preemptive scheduling. Instead, Rust embraces lazy evaluation and cooperative scheduling, allowing developers to meticulously control the yield points to the executor. This level of control, however, introduces complexity in implementing the Future trait for custom types. The intricacies arise because .await can’t be invoked within a non-async function, necessitating the development of a state machine (or similar) for these custom types. This endeavor can be laborious and fraught with potential errors, difficult to maintain, and may prompt developers to opt for BoxFuture<T>, a choice that could compromise performance.

Rust Profiling Essentials with perf

morristai01@gmail.com (Morris) — Mon, 09 Oct 2023 10:04:55 +0800

What is profiling?

A: Sampling the program at specific times, and do some statistics analysis.

It can be one of the following:

Reading Backtraces of the program for every 1000th cycles, and represent in ﬂamegraph.
Reading Backtraces of the program for every 1000th cache miss, and represent in ﬂamegraph.
Reading Backtraces of the program for every 10th memory allocation, and represent in ﬂamegraph.
Get return address of the program for every 10th memory allocation, and show counts for every line.

How to trigger a sample?

Kind of triggers:

Serverless with Rust and Protocol Buffers

morristai01@gmail.com (Morris) — Wed, 08 Mar 2023 19:43:00 +0000

I’ve recently been working on rewriting our small service with Rust for fun. One reason is to see if Rust, as a system programming language, is ready for cloud development in 2023. Another reason is that I wonder how much better it is compared to Python or Java in major cloud and data computing.

💡 There is a lot of discussion over which language to use as a serverless service. In my point of view, dynamic languages like Python lack compile-time checks, which can cause more runtime errors than static languages like Rust or Java.
Another reason is that Python and Java need a runtime process (JVM, CPython) to run your actual code, which means they can’t run natively like compiled Rust does. However, I’m not sure about the performance gap. Furthermore, Rust doesn’t require garbage collection, so ideally, a program’s heap size should fluctuate less than a garbage-collected language when running. That means Lambda or Function underlying infrastructure controller should be able to handle invocation or scaling more easily.
In terms of language paradigms, in my opinion, compared to traditional OOP, the modern ML (meta-language) family has a better design for scalable cloud services.

Make Kafka Schema Easier

morristai01@gmail.com (Morris) — Wed, 26 Jan 2022 10:04:55 +0800

TL;DR: You can find the script on my GitHub repository: kschema-table

Coordinating schema with Data Scientist

As a data engineer, you’ll need to collaborate closely with data scientists/domain experts to design data schemas. The optimal schema will depend heavily on the business domain and how the product is used. For instance, in a cybersecurity context, threat experts and data scientists would likely be the ones designing the data schema. They may work with infrastructure and data teams to define these schemas in a common format like, for example: Avro IDL (Avro Interface Description Language).

SSH Tunneling Summary

morristai01@gmail.com (Morris) — Thu, 10 Dec 2020 10:04:55 +0800

There are three types of SSH port forwarding:

Local Port Forwarding

Forwards a connection from the client host to the SSH server host and then to the destination host port.
Remote Port Forwarding

Forwards a port from the server host to the client host and then to the destination host port.
Dynamic Port Forwarding

Creates a SOCKS proxy server that allows communication across a range of ports.

Roles

Client: Any machine where you can ssh to enable Port Forwarding
SSH Server: A machine that can be SSHed into by Client
Target Server: A machine to which you want to establish a connection, usually to open services on this machine to the outside world.

Notice: both Client and SSH Server can be Target Server, it doesn’t really need three machines to perform Port Forwarding! However, Dynamic Port Forwarding will not be only one Target Server, but it can be dynamically determined.

Practicing bisect in Go

morristai01@gmail.com (Morris) — Tue, 05 May 2020 11:46:56 +0800

In Python, we are used to using bisect to search for elements in the List. In Go, sort.Search() can also provide us with the same purpose. As in Python, the target array is sorted before searching.

1
2


arr := []int{9, 2, 1, 3, 6, 4, 5, 5}
sort. Ints(arr)

After sorting, we use sort.Search(), the first parameter is the search target, and the second is the function of the search condition. Here we input slice: [1 2 3 4 5 5 6 9], find the 5 closest to the left, and the program returns the index 4.

How to mail the movie list from the remote host

morristai01@gmail.com (Morris) — Fri, 10 Jan 2020 17:04:55 +0800

Sometimes I upload videos for my family but don’t know if they already exist on the computer. At this time, I can think of a way to automatically email the titles of the videos already in the video library to myself for reference on a regular basis.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


~/Downloads
➤ cat 2021-12-03-movie_list.txt -r :10
───────┬────────────────────────────────────────────────────────────────────────────────────────────────
 │ File: 2021-12-03-movie_list.txt
───────┼────────────────────────────────────────────────────────────────────────────────────────────────
 1 │ 2021-12-03 Movie List
 2 │
 3 │ CreationTime Name
 4 │ ------------ ----
 5 │ 2021/11/13 下午 05:03:45 紅色通緝令
 6 │ 2021/11/12 下午 04:03:19 高玩殺手
 7 │ 2021/11/12 下午 04:03:17 007生死交戰
 8 │ 2021/11/12 下午 03:57:13 芬奇的旅程
 9 │ 2021/11/12 下午 03:46:12 沙丘
 10 │ 2021/9/21 下午 03:28:33 最佳销售员
───────┴────────────────────────────────────────────────────────────────────────────────────────────────

Gmail low security settings must be turned on

Go to Account Settings
Click Security in the list on the left.
At the bottom of the low security restriction click Turn on access.

In this way, emails can be sent through the smtp.gmail.com api.

Common Python and C++11 usage comparison

morristai01@gmail.com (Morris) — Sun, 12 May 2019 11:46:56 +0800

My Gist

Define Variables

Python
1 2 3

a = 1 b = 0.0 c = 'xxx'

C++

1
2
3
4
5
6
7
8
9


int a = 1;
auto a = 1;
double b = 0.0;
auto b = 0.0;
const char c[] = "xxx"; // C string (char array)
const char* c = "xxx"; // C string (pointer)
std::string c = "xxx";
std::string c("xxx");
std::string c{"xxx"};

Variable Scopes

Python

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


global_var1 = 1
def func(arg):
 local_var = 2
 global global_var2
 global_var2 = 'xxx' # global
if arg:
 local_var2 = 0.5 # scope: function
 ...
 # if arg evaluates to True,
 # local_var2 is still accessible here.
 return False