Learning from Installing Smart Switches

I installed a few smart switches and automated my house lighting recently. It's a long journey and I've learned a lot along the way. As a software engineer, I'm surprised there are a few in common in software engineering and replacing switches. Probably those are common in tackling a problem.

Understand the Goal and Resources

In my daily job and other programming, I mostly focus on completing a product or getting a service online. They're not trivial or simple by nature. We analyze and divide that into minimum viable product, and continue to create tasks and solve the small tasks one by one. Like what we do in the software engineering, I think of what my goal is in this lighting automation project: automate the lights in my house. But it is not every light. I just need to automate lights in the dining room, kitchen, living room, garage, and the doors. This goal is clear and well scoped. It's also clear that I need to replace the existing switches with smart switches, connect the new smart switches to the hub, and configure the rules in the hub to automate the lights.

With the goal, it's also important to understand the resources and priorities. I have a full time job and I like to spend time with my family. That means I cannot devote myself to the project all the time. The good thing is there is no hard deadline. I can think of the small task and work on that one by one. For example, I can just replace one or two switches at a time. With the goal in mind, I need to be flexible and consistent.

Learn And Explore

I feel like there are much to learn in software engineering: language, design, architecture, and process, to name a few. It doesn't mean everybody has to learn all of them now. What I mean is the situations and requirements may change. We need to adapt to that and learn new technology on the way. For example, software engineers who are interested in Mac development won't ignore Apple M1 when it's released. Learning is a nonstop journey. Exploring is how to get to new and unfamiliar areas. It looks simple to replace the switches. But in the end, I need to spend some time in learning some basic wire code, understanding the differences between a regular switch and a smart switch, and understanding how that particular smart switch works in 3-way and 4-way wire. It only becomes possible after I read and explore the way to install the smart switches properly.

Don't Rush

There is always some form of design, development, testing, and release cycle. It's said that if we have a flaw at one stage and carry that down the cycle, it'll cost more to correct it. The later it's fixed, the more expensive it becomes. It's a waste of time and counterproductive if we have to re-work. It also applies when we replace the switches. If we don't get the wires connected right before putting it back, eventually we have to open the plate and correct that. Otherwise, we'll not be able to use the switches. It's important to get things right instead of done quickly. It's critical not to rush to finish it given that it's related to electricity. If we don't do our diligence in the safety, we can get shocked.

Work with the Existing System

It looks great and cool to work on something brand new. That means we have more choices to use new technology. In reality, we most likely add new features to the existing system. It doesn't sound that exciting. But we still need to weigh in how bad the existing system is, whether a from scratch new system can solve those issues, how many resources we need and we get, and whether there are other higher priorities. Replacing the switches is also working with the existing system. It's just too much work to replace the entire wiring in the house. There seems to be some wiring there I don't understand, for example, a white wire is connected to a black wire directly. It's also beyond my ability to try to make it follow the code.

It's fascinating to me that we can borrow the method and approach from one discipline to another. That reminds me that learning is a on-going journey and it can be applied to multiple areas.

Tips on docker-compose

Docker is a technology to create containers. What if we have multiple contains and need them to work together? Docker compose comes to rescue. The configuration is a yaml file and it's easy to follow. I'm going to write down some tips when I learned to use it.

Keep the Docker Container Running

The docker container is supposed to run only one process. The container stops when the process exits. So if we have a process that runs in the forground, it'll most likely exit because there is no console or input. For this kind of container, usually we run the command docker run -ti <image name> to keep the container running. In docker-compose world, you need to put this in the configuration for the service.

tty: true
stdin_open: true

Expose the Network Ports

In running docker, we can use docker -P to expose all the ports specified in the Dockerfile. We can also use docker -p [] to expose the specified ports in the command line. However, docker -P doesn't guarantee to map those ports from the container to the same ports on the host. The mapping is random. We can explicitly set those mapping using docker -p []. docker-compose by default will create a bridge for the specified containers. Depending on how we write it, we can expose the port only to that bridge network or also in the host.

expose:
- "8080"

This will only expose the ports to the linked services.

ports:
- "8080:80"

This also expose the ports on the host. It maps the port 8080 on host to the port 80 on the container.

Connect to the Linked Service in a Container

We specify the service in the configuration. docker-compose will bring up one or multiple container running the same thing for the service. Even if you don't specify the networks, docker-compose still creates a default one for you. You can find all the network by docker network ls. The name of the default network for your docker-compose containers is derived from the directory of the docker-compose.yml file. For example, docker-compose.yml is in the directory 'example'. The default network for those containers is 'example_default'. This may change but this is the behavior right now.

How do you connect to other containers brought up by docker-compose? There are two ways:

  1. Use the service name. For example, in docker-compose.yml, you specify the service like
    services:
    foo:
    image: 'centos:latest'
    bar:
    image: 'ubuntu:latest'

    In the container that runs Centos, you can use bar to connect to the other container that runs Ubuntu.

  2. Use the ip address.
    When you find the network for your containers, you can use docker network inspect <network name> to inspect the details of the network. From there, you can find the ip address of the containers. For example:

    {
        "Name": "example_default",
        "Containers": {
            "4a87be92f610b839c77f0fa87c7bdd13797ca4eac3c2061a4d8e66a1c5e9c867": {
                "Name": "example_foo_1",
                "EndpointID": "0752c01405904b702459560fcc1bc90ef0e24d3e769e8bc46d5edcf06944dba1",
                "MacAddress": "02:42:ac:15:00:03",
                "IPv4Address": "172.21.0.3/16",
                "IPv6Address": ""
            },
            "6244c44cab85df31d684abb5e0b2ce12ecfbef902e13c638d1ab50981b876142": {
                "Name": "example_bar_1",
                "EndpointID": "5b0edb758b080010f8c3fca92e5cb583724c9fefcc23c5148c8c3375517d0093",
                "MacAddress": "02:42:ac:15:00:02",
                "IPv4Address": "172.21.0.2/16",
                "IPv6Address": ""
            }
        },

By far, those are the things I think it's useful when using docker-compose. I spent some time on searching online. I hope this can help you to try out the way to manage containers.

What Do We Need to Know about DNS

Not every programmer needs to know DNS. We tend to take it for granted. It's there. It's working. As we are moving to the cloud and Devops, it's inevitable that we need to deal with network configuration. It's important to understand some basics of DNS.

This is the place to add a custom domain to a web in Azure. It provides A record and CNAME record. What record type should I use?

Add a custom domain for Azure app service
Add a custom domain for Azure app service

What's DNS

If you still have some memory about basic network. IP address is what we use to locate a service on the Internet. But how many IP addresses do you remember? We rarely use the IP address directly. Those are hard-to-remember long numbers. Instead, we remember domain names. What is a domain name? When you type in the browser address bar like google.com, that google.com is the domain name. But we use the IP address. How do we find the IP address by the domain name? That comes the DNS (domain name systems). It resolves the domain name to the IP address.

DNS Hierarchy

You don't need to worry about setting up DNS for your network at home. When the modem connects, it will set up the DNS. Usually it is the ISP DNS. Does it know all the domains? No. It inquires other kinds of DNS servers. These DNS servers together resolve the domain name to the IP address.

DNS servers
dns server

At the top, it is the root server. The root server doesn't know what the domain name map to. But it knows what top level domain server (TLD) is. The top level domain is something like .com, .net etc. For example, google.com goes to the. com top level domain server. TLD server goes to authoritative servers to find the IP address.

The ISP server can cache the result. So it doesn't waste time to go to root servers and other servers.

DNS Records

DNS records are stored in the DNS servers. The record contains information about how to resolve the DNS inquiry. For example, if there is a record like [google.com 172.217.3.174], and when you inquire google.com, it returns the IP address.

There are three types I want to cover here. CNAME (Canonical Name record). It sets up a domain name to its canonical name. In another word, it creates an alias to the canonical name. For example, for this record [foo.com bar.com], when you inquire foo.com, it'll return the canonical name bar.com. The CNAME record doesn't contain the IP address.

The second type of record is A or AAA. It maps from the the domain name to the IP address. Record A maps to the IPv4 address and AAA maps to the IPv6 address.

The third type of record is TXT. It allows to add any human readable text to the domain. You can also add machine readable contents too. What is the content? It's up to you. Consider this is a note to the domain.

How to Get the IP Address

When the browser needs to get the IP address for the domain in the address bar, it'll send the inquiry to the DNS server configured for the network. The DNS server will ask for that following the DNS hierachy. If the domain in the address bar is an alias, it'll get a CNAME record. It then restart the process again with the canonical name. This time it'll receive the A record or AAA record which contains the IP address.

By now, hopefully you have a clearer answer to what to choose when you try to add a new custom donmain to the web in Azure. The DNS is the infrastracture of the Internet. We all should know what the underlying works when developing in the cloud era. Cloud computing has already taken care of a lot of stuffs. We still need to know the basics to be better working in the cloud.

Get Started with Scheme

Scheme is one of the three major dialects of the programming language Lisp. It has many similar syntax as Lisp but features minimalism. In this post, I'll summarize basic Scheme syntax after I read this book to get started with Scheme. That should be helpful to get started since I already have some programming experience.

Everything Meaningful Is in a List Form

The basic syntax is in the form (obj1 obj2 ...) or(expression1 expression2 ...). For example, (1 2 3) is a list that contains 1, 2, and 3. Each element can be of different types. So it can contain strings too. For example, (1 2 'hello'). If it is a procedure call, the first expression is the procedure name. For example (+ 1 2). The first element is +. It's the sum operator. It adds the numbers from the remaining of the list.

We can nest expressions. Each expression itself can be in the list form. That being said, the first one can be in the list form which returns a procedure. And then apply the remaining elements to the returned procedure.

You can have a single symbol or value too. For example "Hello" will generate "Hello". And 2 will generate 2. But if you want to have a meaningful operation, you should use the list form.

Basic Operations

These are the basic operations

  1. (quote expression)

    As I said before, the data and procedure call are in the same form. quote will force to treat the remaining as data. For example, (+ 1 2) is to calculate the sum of 1 and 2, which it generates 3. (quote (+ 1 2)) will always treat (+ 1 2) as data. + becomes the symbol instead of the operation. So it generates (+ 1 2) instead of doing the calculation. We can use ' to replace quote for abbreviation. So '(+ 1 2) is the same as (quote (+ 1 2)).

  2. (car list-expression) returns the first element in the list

    (car '(1 2 3)) => 1

  3. (cdr list-expression) returns the list that contains the elements except the first one.

    (cdr '(1 2 3)) => (2 3)

  4. (cons obj1 obj2) generates a new pair in which obj1 is the first element and the remaining are from obj2.

    It can generate a proper list or an improper list. For example,
    (cons 'a 'b) => (a . b) ; an improper list
    (cons 'a '(b c)) => (a b c) ; a proper list

Variable, Expression and Procedure

The syntax to define a lambda expression is

(lambda (var ...) body1 body2 ...)

The syntax to define a top level variable or procedure is

(define var expression)

The syntax to create a variable at the local scope is

(let ((var expression) ...) body1 body2 ...)

We can use both define and let to create a variable or a procedure.
For example, (define count 0) creates a variable count with the value 0.
(let count 0) also creates a variable count with the value 0.
To define a procedure, it's in the form

(define sum
    (lambda (x y)
        (+ x y)))

This creates a procedure sum at the top level. It sums up the two given arguments.

The difference between define and let is that define creates the variable or procedure at the top level. Any other code can reference to it. let creates it inside the let scope. That means, the variable or the procedure isn't available outside of let code.
We can nest let statements. In this case, the variable or procedure with the same name will shadow the one from outer scope. Here is the code to demonstrate it.

(define var 1)
(let ((var 2))
    (let ((var 3))
        (display var)
        (newline))
    (display var)
    (newline))
(display var)
(newline)

(display) is used to show the argument on the output. (newline) shows a new line. The output of this code is

3
2
1

The variable var in the inner scope shadows the one from outer scope.

Conditional Expression

We can do a if check in Scheme. The syntax is

(if test consequent alternative)

The test can be and, or, and not. The syntax is

(and expression ...)

(or expression ...)

(not expression)

Similarly to if, we can also do switch

(cond (test expression) ... (else expression))

There are special symbol. #t means true and #f means false.

Report errors

(assertion-violation symbol-of-scope message what-violates)

Set Assignment

(set! symbol value)

Improper List

The definition of a proper list uses recursion. A proper list is the list of which the (cdr) is a proper list. And an empty list is a proper list. If a list isn't a proper list, it's an improper list. An improper list is denoted by a '.'. For example (a . b) is an improper list and (a b) is a proper list. Below is an example in the code

(cons 'a 'b) => (a . b)
(cdr (a . b)) => b ; b is not a list
(cons 'a '(b)) => (a b)
(cdr (a b)) => (b); (b) is a list

These are the basic syntax and building block of Scheme. With that we can go on to the advanced Scheme and start writing programs.

Make It Your Habit to Fix Build Warnings

Warnings are not errors. Why do we bother to spend time fixing them? It feels so fast to get it compiled and run before looking into the warnings. It's because warnings may reveal that the code is not what you mean. It should be your habit to fix build warnings.

Errors or Warnings

Build errors are those that prevent you from getting a program or running it. Like it or not, you need to fix all the build errors. Warnings, on the other hand, don't have those restrictions. Anybody can ignore them and run the program. Luckily, the compiler usually has an option or switch to treat warnings as errors. By doing so, anybody has to address those warnings before they can continue to run the program. Having the compiler to treat warnings as errors is the only way to enforce everybody to fix the warnings too.

I'll show it how to do it in .Net Core and illustrate the importance of warnings by some examples.

Enable Warnings as Errors

There are two ways to enable warnings as errors.

The first one is to set the property TreatWarningsAsErrors in the project file. for example, in the csproj file,

<PropertyGroup>
  <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>

On the other hand, <NoWarn> is used to exclude certain warnings

<NoWarn>$(NoWarn);CS0168;CS0219</NoWarn>

<WarningLevel> is used to set the warning level. The higher the number is, the more less servere warnings it'll report. The default value is 4.

Second, use the compiler option -warnaserror. For example,

dotnet build -warnaserror example.csproj

This will turn all warnings to errors. You can set specific warning with -warnaserror like -warnaserror:642,649. -nowarn on the other hand, disables certain warnings.

-warn is used to set the warning level.

The Warning Examples

  1. warning CS4014: Because this call is not awaited, execution of the current method continues before the call is completed. Consider applying the 'await' operator to the result of the call.

    This occurs when you forget to use await on any async call. It reveals a mistake in the code even though it's not a build error. There may be a subtle bug when the program runs. It's not always happening which makes it much difficult to reproduce the bug and diagnose. For example

    using (FileStream fs = File.Create(path))
    {
         fs.WriteAsync(bytes, 0, byteSize, CancellationToken.None);
    } 

    Without await on fs.WriteAsync, it's possible that fs is being disposed while WriteAsync is still running. It all depends on the scheduling. So it's quite difficult to diagnose.

  2. warning CS1717: Assignment made to same variable; did you mean to assign something else?

    I'm sure you don't intend to assign to the same variable. It could be a typo. But you may not notice it in a quick glimpse. For example

    class Test
    {
        private int count;
        public Test(int count)
        {
            count = count;
        }
    }
    
    By treating warnings as errors, you're forced to look at the message and look at what's going on.

Exceptions

No all warnings indicate mistakes. Spend some time thinking about that. If you can justify it, you can suppress the warnings. One way of doing that is to use <NoWarn> in the project file or -nowarn in the compiler option. I don't recommend it because it applies to the whole project. I suggest to use pragma warning to disable the warning. Remember to enable the warning afterwards. For example


#pragma warning disable CS3021
//code
#pragma warning restore CS3021

Take away

Always set the flag to treat warnings as errors in your build or compiler. Always fix warnings except you can justify not to. Always explicitly surpress certain warnings before the line and restore it afterward.