Working from Home + Corporate VPN

Hey everyone! today I’d like to tell you a small story about a guy that’s been stuck working from home for more than a year and is suffering from constant VPN issues. This guy is me!

TLDR: can’t access corporate resources? use static routing to redirect connections through the corporate VPN.

Ok, so the story goes like this: I work at a Big-Corp & my team has a couple of VMs deployed on Azure which are governed by an internal service that makes sure that nobody can access them outside the corporate VPN. The service thus a bunch more things, but the gist is that one can’t alter the network security group (“firewall” rules) such that outsiders can access the machine.

That’s pretty neat, but problematic when you work from home. Why? not all network originating from our machine goes through the corporate network. In other words, when I try to access any resource that belongs to my team, the IP that’s being used is the one that belongs to my ISP –> forbidden.

How do I know that? my laptop’s routing table. The routing table is in charge of mapping the network routes for a given destination IP. On my macOS machine the routing table looks something like this:

$ netstat -rn
Routing tables

Internet:
Destination Gateway Flags Netif Expire
default 10.0.0.1 UGSc en0
127 127.0.0.1 UCS lo0
127.0.0.1 127.0.0.1 UH lo0

I truncated some values, but the gist is: any traffic going to 127.0.0.0/8 (255.0.0.0) -> use the lo0 interface, and everything else go to the default, which is en0 or 10.0.0.1. If I turn on the VPN, a new virtual network device appears (a.k.a: tun device) and many (many!) more routes show up:

$ netstat -rn
Routing tables

Internet:
Destination Gateway Flags Netif Expire
default 10.0.0.1 UGSc en0
127 127.0.0.1 UCS lo0
127.0.0.1 127.0.0.1 UH lo0
<corporate-ip-1> 10.10.5.125 UGHS utun2
<corporate-ip-2> 10.10.5.125 UGHS utun2
...
<corporate-ip-500> 10.10.5.125 UGHS utun2

What basically happens is that the VPN client connects to the vpn server, creates a secure tunnel between my computer and the VPN server, fetches a policy that dictates which IPs and subnets should go through the corporate network and configures my machine to send such traffic through the virtual interface.

When a packet hits the virtual interface, it gets encrypted and sent over the public internet to the VPN server. The VPN server has two “legs”: one in the corporate network and one in the public network. Once a client connects, every (verified) packet that arrives to the VPN server on the public internet gets sent to the corporate network.

In our case, we have an azure resource that can only be accessed through the corporate network.
we want to access a specific azure resource that is not configured in the VPN policy. There are two solutions: either fix it locally or use a jumpbox.

A jumpbox is a machine that has access to both networks: the public one and the corporate one. I can connect to it (using ssh, rdp, whatever) and than connect to the remote resource. SSH even has a flag for that (-J or ProxyJump):

# We connect to the jumpbox than the jumpbox connects to the remote-pc,
# and proxying the traffic between our machine and the remote.
$ ssh -J otheruser@jumpbox user@remote-pc

Hack, why should we need another PC? why not just route traffic to the VPN? should be easy. well, it is, all we need to do is a simple static route:

$  sudo route -n add -net <remote-pc> <vpn-gateway-ip>

That’s easy enough, but I honestly don’t want to add a static route everytime I want to connect to a remote resource. ITS A PAIN IN THE ASS! Why not write script to do that for me?

Well, I need a few things:

First, I need a way to extract the gateway IP address. the problem? GlobalConnect uses a different IP and with a different interface name everytime it connects!

Second, I want a robust script that has the same completion as ssh does.

Let’s tackle the first issue: I guess that GlobalConnect has logs somewhere or a configuration that specifies the interface and ip address it used to create the tun device. in macOS this stuff should sit at /Library. I grepped for the VPN server address and viola!

/Library/Preferences/com.paloaltonetworks.GlobalProtect.settings.plist contains the VPN configuration:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Palo Alto Networks</key>
<dict>
<key>GlobalProtect</key>
<dict>
<key>PanGPS</key>
<dict>
... many keys ...
<key>PreferredIPV6_11de985a54adccec223de2b24075693f</key>
<string>3b23:110:8:6:6000::5</string>
<key>PreferredIPV6_234d2be7acce021cd9e49f5c5a7bc41</key>
<string>3b23:110:68:41:6000::9</string>
<key>PreferredIPV6_93bb97413136ce437a86418851ac5e99</key>
<string>2404:f801:50:a:6000::84</string>
<key>PreferredIPV6_d1ded811e3311a45e766352fca97c7cb</key>
<string>3b23:110:20:10:6000::147</string>
<key>PreferredIP_11de985a54adccec223de2b24075693f</key>
<string>10.10.1.35</string>
<key>PreferredIP_234d2be7acce021cd9e49f5c5a7bc41</key>
<string>10.10.5.125</string>
<key>PreferredIP_93bb97413136ce437a86418851ac5e99</key>
<string>10.10.1.38</string>
<key>PreferredIP_d1ded811e3311a45e766352fca97c7cb</key>
<string>10.10.1.120</string>
</dict>
... many keys ...
</plist>

God! this is a piece of shit. I hate parsing xml… maybe there’s something that can parse plist files? YESSSS!!! /usr/libexec/PlistBuddy does just that! it’s like jq but for plist!

$ GP_CONFIG=/Library/Preferences/com.paloaltonetworks.GlobalProtect.settings.plist
$ /usr/libexec/PlistBuddy -c "print 'Palo Alto Networks':GlobalProtect:PanGPS" "$GP_CONFIG" | \
awk -F'=' '/PreferredIP_/ { print $2 }' | \
tr -d '[:blank:]' | \
xargs -I {} fish -c "ifconfig -a | awk '/inet {}/ { print \$2 }'"

Turns out there’s another way. I also grepped the vpn gateway IP and found it at /Library/Logs/PaloAltoNetworks/GlobalProtect/network/config/itf-install.sh.

GlobalConnect writes down all the configuration to the logs directory, including the scripts it uses to configure the routes. the itf-install.sh contains this:

/sbin/ifconfig utun2 10.10.5.125 10.10.5.125 netmask 255.255.255.255 up
/sbin/ifconfig utun2 inet6 2a01:110:68:41:8000::9/128 up

Awesome! I can just awk it, but I do need root permissions :(

$ ITF_INSTALL=/Library/Logs/PaloAltoNetworks/GlobalProtect/network/config/itf-install.sh
$ sudo awk '/\/sbin\/ifconfig [^ ]+ [0-9]/ { print $3 }' "$ITF_INSTALL"

Anyway, once that’s working, what about completion? well, in fish shell this is pretty straight forward. All I had to do is find the completion file for ssh and just copy it for my script. I called my script vssh and the ssh completion file sat here: /opt/local/share/fish/completions/ssh.fish. I copied it to ~/.config/fish/completions, replaced ssh with vssh and viola! I had tab completions :)

Let’s start writing down our script… the skeleton should look like this:

#!/usr/bin/env fish

set remote <remote-pc>

# get the ip address of the vpn device, using the first method
set gateway (get_global_protect_interface_ipaddr_from_config)

# if first method failed, try the second method
if test -z "$gateway"
echo "need root privileges in order to find the vpn gateway"
set gateway (get_global_protect_interface_ipaddr_from_install_script)
end

# test that we have a gateway
if ! ifconfig -a | grep "$gateway" &>/dev/null
echo "couldn't find any interface bound to $gateway"
echo "maybe the vpn client is not connected?"
exit 1
end

if <remote-pc-default-gateway> != <vpn-gateway>
add_static_route <remote-pc> <vpn-gateway> || exit 1
end

ssh $argv

this is pretty straight forward, but two things are missing: how do I find out the hostname if I’m passing arbitrary ssh args? and second, how will this work with ssh-config?

ssh -G to the rescue! the -G flag causes ssh to print its configuration after evaluating Host and Match blocks. I can use it to find out the actual hostname that ssh is going to use in order to connect to the remote host!

set remote (ssh -G $argv 2>/dev/null | awk '/^hostname/ { print $2 }')

cool, right?! That’s all folks! I’ve uploaded the whole script to my GitHub. as always, feel free to comment :)

AX3600 - Post #1

In my previous post I gave you an intro about AX3600 and how awesome it is. This post will focus on how to gain persistent ssh on the device.

First off, I recommend following the OpenWRT effort on porting AX3600: Adding OpenWrt support for Xiaomi AX3600 For Developers. Until then, follow this guide.

All the scripts, firmwares, configuration files, etc’ mentioned in this post are available on GitHub: odedlaz/ax3600-files. Please feel free to open issues or better yet - pull requests!!

gain SSH access

All you need to know is that early versions of the router had a command injection vulnerability that is now closed. In order to gain ssh access, you first need to downgrade your router’s firmware then run the commands listed in OpenWRT’s page. But what is the command injection? taking closer look at the source…

We want to extract the firmware. Thankfully, there’s a tool just for that: ubi_reader.
Follow the instruction and get it up and running. Once you do, download the firmware and run:

$ ubireader_extract_images -o xioamifw -w miwifi_r3600_firmware_5da25_1.0.17.bin
$ unsquashfs -f -d xiaomifw xioamifw/miwifi_r3600_firmware_5da25_1.0.17.bin/img-1696626347_vol-ubi_rootfs.ubifs

The command injection URL looks like this:

http://192.168.31.1/cgi-bin/luci/;stok=<STOK>/api/misystem/set_config_iotdev?bssid=gallifrey&user_id=doctor&ssid=-h%0Anvram%20set%20ssh%5Fen%3D1%0A

How does it work? let’s find set_config_iotdev:

$ grep! -R "set_config_iotdev"
xioamifw/miwifi_r3600_firmware_5da25_1.0.17.bin/output/usr/lib64/lua/luci/controller/api/misystem.lua: entry({"api", "misystem", "set_config_iotdev"}, call("setConfigIotDev"), (""), 221)

Ok, it looks like there’s a file called misystem.lua that contains all the controller apis. Also, the set_config_iotdev parameter is tied to the setConfigIotDev. easy enough… let’s look at it:

function setConfigIotDev()
local XQFunction = require("xiaoqiang.common.XQFunction")
local LuciUtil = require("luci.util")
local result = {
["code"] = 0
}

local ssid = LuciHttp.formvalue("ssid")
local bssid = LuciHttp.formvalue("bssid")
local uid = LuciHttp.formvalue("user_id")

XQLog.log(debug_level, "ssid = "..ssid)
XQLog.log(debug_level, "bssid = "..bssid)
XQLog.log(debug_level, "uid = "..uid)
if XQFunction.isStrNil(ssid)
or XQFunction.isStrNil(bssid)
or XQFunction.isStrNil(uid) then
result.code = 1523
end
if result.code ~= 0 then
result["msg"] = XQErrorUtil.getErrorMessage(result.code)
else
XQFunction.forkExec("connect -s "..ssid.." -b "..bssid.. " -u "..uid)
end
LuciHttp.write_json(result)
end

Very straight forward and classic! look at the line starting with forkExec: there’s no input validation, thus parameters supplied to the api are used on the router itself and passed directly to exec. The injection basically add a \n to the end of the last parameter, following by the command we really want to run, which is nvram set ssh_en=1.

By the way, on later firmwares the misystem.lua is compiled and encrypted. While trying to figure out how to decrypt it I came across a great deck called Exploit (Almost) all Xiaomi Routers Using Logical Bugs which you might like.

Enough! we just want to gain SSH! Let’s downgrade. It’s pretty easy, all you need to do is:

  1. access the setting page: http://192.168.31.1/cgi-bin/luci/;stok=<STOK>/web/setting/upgrade
  2. press “Manual upgrade” (I’ve got Google Translate turned on)
  3. upload the firmware (miwifi_r3600_firmware_5da25_1.0.17.bin). I backed it up since I’m a bit paranoid. You can find it under the firmwares directory in my repo or the official link.

Afterwards, you can follow the guide supplied by OpenWRT or just import the Postman collection I created. Don’t forget to update the stok variable in the collection. You can grab it by logging in to the router web interface and getting the value of “stok=” from the URL.

gain persistent SSH

A guy called didiaoing wrote a nice tutorial which explains how to modify the routers bdata partition in order to persistenly turn on ssh. The problem? it’s manual, error prone and in chinese.

Enough talkin’! let’s get flashin’!

Clone my repo and create a bdata directory:

$ git clone https://github.com/odedlaz/ax3600-files.git
$ cd ax3600-files
$ mkdir -p bdata

The router’s uses UBIFS as it’s filesystem. UBIFS workes on top of UBI, which in turn works on top of MTD.

You can ssh to your router and see the mtd partitions for yourself, they’re all there for the taking:

$ cat /proc/mtd
dev: size erasesize name
mtd0: 00100000 00020000 "0:SBL1"
mtd1: 00100000 00020000 "0:MIBIB"
mtd2: 00300000 00020000 "0:QSEE"
mtd3: 00080000 00020000 "0:DEVCFG"
mtd4: 00080000 00020000 "0:RPM"
mtd5: 00080000 00020000 "0:CDT"
mtd6: 00080000 00020000 "0:APPSBLENV"
mtd7: 00100000 00020000 "0:APPSBL"
mtd8: 00080000 00020000 "0:ART"
mtd9: 00080000 00020000 "bdata"
mtd10: 00080000 00020000 "crash"
mtd11: 00080000 00020000 "crash_syslog"
mtd12: 023c0000 00020000 "rootfs"
mtd13: 023c0000 00020000 "rootfs_1"
mtd14: 01ec0000 00020000 "overlay"
mtd15: 00080000 00020000 "rsvd0"
mtd16: 0041e000 0001f000 "kernel"
mtd17: 016c4000 0001f000 "ubi_rootfs"
mtd18: 01876000 0001f000 "data"

We’re interested in the bdata partition. go ahead and dump it:

bash
$ nanddump -f /tmp/bdata_mtd9.img /dev/mtd9

then copy it to your machine. If you’re using linux, then the following command should work:

$ scp [email protected]:/tmp/bdata_mtd9.img /path/to/ax3600-files/bdata

If you open it up in a hex editor, you’ll notice a few weird bytes in the beginning then a few ASCII encoded strings, a looooot of zeroes and some other bytes in the end. the first four bytes are the CRC32 checksum for the bdata partition and the strings indicate the devices configuration. you probably recognize some of them, since you’ve updated them when turning on ssh on the device!

You can also look at the header by running the header.py provided script. just cd to the scripts directory and issue the following command:

$ ./header.py extract ../bdata/bdata_mtd9.img
CRC32: 17 27 E1 B2
color: 101
CountryCode: CN
SN: 11233/A0C123456
model: R3600
miot_did: 112233444
miot_key: aap0blsq5aQbVFmi
telnet_en: 0
ssh_en: 0
uart_en: 0
wl0_ssid: Xiaomi_EA69_5G
wl1_ssid: Xiaomi_EA69
wl2_ssid: Xiaomi_EA69

We want to update these values to enable ssh on boot. Since dropbear is not turned on by default, you need to enable telnet as well. Actually, if we’re going to mess with the router, why not turn on the router’s serial port so you could flash the entire thing if you break it? Darell Tan’s got a nice post on customizing the firmware through serial. great read!

Moreover, in the next post I’ll write about the weird stuff I found on the router. Some of these stuff only run if the router’s Country Code is set to CN, so why not change it to US? this is not random! when you change the country code to something other than CN, the router behaves a bit differently. for instance, it performs online checks against google & microsoft instead of baidu & taobao. both are configured in /etc/config/system and accessed by running uci -q get system.netdt.world_domain and uci -q get system.netdt.cn_domain respectively. Don’t believe me? take a look at the check_gateway function in usr/sbin/pppoe-check.

Once you update the headers, you also need to update the checksum. Don’t worry, header.py does it all for you.
First let’s do a test run to make sure we don’t break anything:

$ ./header.py modify --test ../bdata/bdata_mtd9.img ../bdata/bdata_mtd9.img.modified
successfully re-assembled header without modifications

If you got any response other than the above, please open an issue! if everything went fine, you can run:

$ ./header.py modify --country US ../bdata/bdata_mtd9.img ../bdata/bdata_mtd9.img.modified
$ ./header.py extract ../bdata/bdata_mtd9.img
CRC32: 27 21 A1 D2
color: 101
CountryCode: US
SN: 11233/A0C123456
model: R3600
miot_did: 112233444
miot_key: aap0blsq5aQbVFmi
telnet_en: 1
ssh_en: 1
uart_en: 1
wl0_ssid: Xiaomi_EA69_5G
wl1_ssid: Xiaomi_EA69
wl2_ssid: Xiaomi_EA69
boot_wait: on

You’ll notice that the checksum is now updated and CountryCode, telnet_en, ssh_en, uart_en were all changed. Moreover, a new boot_wait config has been added.

Unfortunately, the bdata partition is read only. You need to make it writable prior to flashing the bdata partition or you’ll get a readonly error:

$ mtd write /tmp/bdata_mtd9.img bdata
Could not open mtd device: bdata
Can't open device for writing!

How? flash a new crash partition. Doing so opens the bdata partition for writing. I was (and still am) quite afraid to do that, since I have no clue what that crash partition contains. A random guy named barnamacko uploaded a crash partition that works on the OpenWRT forum. Flashing an unknown binary from an unknown source on your home router? why not! on a more serious note, I wasn’t keen to do that. I’m not sure what that binary contains, but I do know that it doesn’t modify the filesystem (I checked).

I recommend you physically connect to the router before performing any of these commands. Some people lost Wi-Fi connectivity when they flashed the crash partition (don’t worry, after removing it Wi-Fi works again!)

Go ahead and run the following on your computer:

$ scp ../crash/crash_unlock.img [email protected]:/tmp

Then login to the router and flash it:

$ mtd write /tmp/crash_unlock.img crash
$ reboot

post reboot, upload the modified bdata partition you created:

$ scp ../bdata/bdata_mtd9.img.modified [email protected]:/tmp

and flash it:

$ mtd write /tmp/bdata_mtd9.img.modified bdata
$ reboot

After reboot, login to your router again and remove the crash partition:

$ mtd erase crash

Then perform a factory reset on the device by surfing to: http://192.168.31.1/cgi-bin/luci/;stok=<STOK>/web/setting/upgrade. This step is important because otherwise you might bump into upgrade verification issues.

After the reset, you can upgrade to the latest firmware! If you do so, you’ll lose ssh connectivity. Don’t worry, you can connect to the device via telnet:

$ telnet 192.168.31.1 23
XiaoQiang login: root
Password:

Oh snap! what’s the password? well, after you patched the router and upgraded the firmware, the ssh password you set beforehand got reset. dear didiaoing created a website that generates the password for you! all you need to do is fill in the router’s SN number and get the default password. There’s nothing bad about sending your routers SN to a random website which performs all the calculation on the backend, right? I had to do some digging in order to find the algorithm that’s being usedd to generate those :)

thankfully, I found another chinese guy that goes by the alias zhoujiazhao, that wrote the algorithm in php. who does that these days?! PHP? really? me to the rescue, I ported it to python. Just run calc_passwd and you’re good to go:

$ ./calc_passwd.py ../bdata/bdata_mtd9.img.modified
1b6e63b5

By the way, I haven’t encountered password issues, only read about them online. Once I got ssh acsess and reset the password, I flashed the international firmware and was able to connect with telnet.

Once you’ve got shell access, you can turn on ssh by running postman steps (4) to (7), or just reset the password to admin:

$ sed -i 's/channel=.*/channel=\"debug\"/g' /etc/init.d/dropbear
$ /etc/init.d/dropbear start
$ echo -e 'admin\nadmin' | passwd root

Moreover, if you do decide to flash the international firmware, you probably want to change the interface language to English:

$ uci set luci.main.lang='en'
$ uci commit

Don’t forget to turn off telnet after you verify ssh is working:

$ nvram set telnet_en=0
$ nvram commit

That’s it folks! I hope you won’t need to do any of this once the guys at OpenWRT finish porting it to AX3600 :)

AX3600 - Post #0

Wow. last time I wrote anything was ~2 years ago. I actually spent a whole day on getting the blog to working state!
Anyhow, I’m back and I’ve got some cool stuff to share. Hopefully I’ll keep posting things regularly…

So… what are we going to talk about today? A router! and not just any router - Mi AIoT Router AX3600!

I have at least three posts for this topic:

  1. Why I bought it & my journey into getting persistent ssh on the device
  2. All the weird stuff that Xiaomi put on the router and how to disable them
  3. How to turn off all wierd stuff Xiaomi put on it

Let’s get this started, shall we?

The Good

AX3600 is, to the best of my knowledge, the most affordable router with carrier-grade hardware. The AX3600 runs on Qualcomms’s Networking Pro 600 (IPQ8071 SoC) which would be quite expensive if bought from other reputable manufacturers.

Xiaomi did something incredible: they packaged all this power into a nice housing and priced it undress 100$. Yes, you read that correctly - a carrier-grade, WiFi 6 router that costs under 100$. I actually bought it from AliExpress for 130$, including shipping!

The Bad

Xiaomi didn’t have a global version of the router so the whole interface would be in chinese. That’s a bummer, but I hoped that the guys at OpenWRT would work on porting that beautiful platform to the AX3600. There was a lot of work being done on the router a few months ago so I thought that a port is around the corner. Fast forwardt to November 2020 and it looks like it might take a while.

On the flip side, Xiaomi forked OpenWRT and the guys there found a command injection vulnerability that allowed them to start an ssh server (dropbear) and gain root on the device.

The Ugly

I guess most of you heard that some chinese manufacturers spy on you. right? There was a lot of talk about Huawei which got eventually banned from being used by any U.S federal authority. The verge actually wrote a piece about that a few months ago.

I initially thought this is BS and that Xiaomi wouldn’t do that. After dissecting the firmware for the last few weeks I can honestly say that I’m not so sure anymore.

Now What?

I wrote a few tools (odedlaz/ax3600-files) to help me customize the router’s firmware. I’ll write about them and the steps needed to gain persistent ssh in the first technical post in the series.

Stay tuned!

Getting an Accurate Weather Forecast using Wi-Fi Position System

I’m travelling a lot with my laptop, and having an accurate weather forecast on-the-go is very convenient.

I’m using Fedora as my main (and only) OS, along with i3wm and polybar. Previously I used GNOME 3 which had a few weather widgets. The most feature-rich was OpenWeather, which worked pretty well.

polybar doesn’t have a native weather widget and the one’s at x70b1/polybar-scripts suffered from the same drawbacks that OpenWeather did: location had to be manually specified or was ip-geolocation based (which is not accurate enough)

How do I get an accurate location without a GPS? WiPS to the rescue!

Wi-Fi Positioning System

Did you ever notice that you can get accurate location data when your GPS is turned off? The trick is to measure Wi-Fi signal intensity in order to triangulate your location.

Google, for instance, tracks open access points when you walk around and upload these along with your GPS location. When your GPS is turned off, they can use the access points around you to calculate your GPS coordinates. Cool right?

Lifewire published an explanation about Wi-Fi triangulation a few months ago, and Wikipedia has a nice article on the subject as well.

Getting Accurate Geolocation

Google Maps has a neat Geolocation API that gives accurate location data if supplied with the MAC addresses of surrounding access points.

I can use iw to extract these by running:

$ iw <iface> scan | grep -io "[0-9A-F]\{2\}\(:[0-9A-F]\{2\}\)\{5\}" | uniq

But how the heck do I get the wireless interfaces? well, there’s /proc/net/wireless for that -

import re

def get_wireless_interfaces():
iface_rx = re.compile('^(.*): ')

# I love one-liners. Hope that doesn't look like perl :)
# - open() returns a read-only file wrapper that is iterable
# - map() executes re.match() on each line returned from open()
# - 'yield from' is used to delegate the generator were creating to another generator
yield from (m.group(1) for m
in map(iface_rx.match, open('/proc/net/wireless'))
if m)

Now, I need to to run iw. Running system commands from python is a pain in the ass.
unless you use the mighty sh package, that is!

import sh

def get_access_points_addresses():
rx = re.compile('([0-9A-F]{2}(:[0-9A-F]{2}){5})', re.IGNORECASE)
for iface in get_wireless_interfaces():
matches = rx.findall(sh.iw(iface, "scan").stdout.decode('utf-8'))
yield from (x[0].strip().lower() for x in matches)

And now all that’s left is to kindly ask google to give us our location.
urllib is a pain, so I’m using requests instead.

import requests

def get_geolocation():
# "considerIP" flags indicates if google should fallback to ip-based
# geolocation if it can't find your ip using the given access points.
data = {"considerIp": True,
"wifiAccessPoints": [{"macAddress": hwaddr}
for hwaddr in set(get_access_points())] }



r = requests.post("https://www.googleapis.com/geolocation/v1/geolocate",
json=data,
params={"key": GOOGLE_MAPS_API_ACCESS_KEY })

r.raise_for_status()

location = r.json()["location"]
return location["lng"], location["lat"]

Getting Weather Forecast

OpenWeather widget used OpenWeatherMap API to get up-to-date weather forecast, so I used it as well.

import requests
def get_weather_forecast(longitude, latitude):
# I'm living in a country that uses a unit system that makes sense.
params = {"units": "metric",
"appid": OPEN_WEATHER_MAP_API_KEY,
"lat": latitude, "lon": longitude}

resp = requests.get("https://api.openweathermap.org/data/2.5/weather",
params=params)

resp.raise_for_status()

return resp.json()

And viola!

longitude, latitude = get_geolocation()
forecast = get_weather_forecast(longitude, latitude)
print((f"{forecast['name']} "
f"{forecast['main']['temp']:.0f}°C "
f"[{forecast['weather'][0]['main']}]"))

# Tel-Aviv 18°C [Clear]

You're not a Java Developer

You are not a “Java Developer”.
You are a Software Developer who is capable of writing software in Java.

You don’t want to hire people that call themselves “Java Developers”.
You want to hire devs that either have extensive knowledge in Java (and are also good software developers), or hire superb developers that don’t, but can close the gap quickly.

Java, or any other language for that matter, is just syntax. Learning a language is a long process, which consists of:

  • Understanding how to write good software with it
  • Understanding How the language works under the hood
  • Understanding What are the language’s limitations
  • Understanding which problems are good to solve with it, and more importantly - which aren’t.
  • Learning the eco-system: libraries, tooling, etc’

I can’t stress that enough.

The language does not define you. What does?

  • Your ability to make the right architectural decisions
  • Your ability to take responsibility for your work
  • Your ability to grasp new ideas
  • Your ability to learn new languages & technologies quickly
  • Your passion for software crafstmanship & understanding that you have to constantly learn in order to be good at it.

You don’t want to be an “X Developer”, because that means that you are only expected to know X. IMO, that means that you have only one tool in your toolbox which you use to solve every problem.

You want to learn new languages, new operation systems, new tools, new techniques.
Each one teaches you new ways to reason about & solve problems.

Java is very good language for a given set of problems. It’s a very bad language for others. Furthermore, Java has it’s own view on how to solve problems. Haskell, for example, look at & solves problems in a completely different way.

The “I’m an X Developer” kind of people look at (and reason about) problems from the wrong angle. Instead of looking at a problem and searching for the right tool to solve it, They use the tools they feel comfortable with in order to solve it.

Don’t get me wrong, being proficient in a specific domain (which might be a language and its ecosystem) is important, but the domain does not define you and you should not limit yourself to only one.

The way I see it, there is no such thing as a “Java Developer”, there’s a developer that is proficient in Java. This subtlety is important.

Migrating Fedora from BIOS to UEFI

Let me tell you a story.

This is not a sad story, but a geeky one.

A story about a developer that was told it’s impossible to migrate his Fedora OS from BIOS to UEFI, and against all odds, succeeded.

A few months ago I started working at a new place and got a shiny Dell XPS 9560.

The spec was amazing: Top of the line CPU, GPU, 4k screen and even 32gb of RAM!

But the issues.. oh… the issues. Thank god most of them are solvable by a simple firmare upgrade. The rest are GPU issues which led me to disable the embedded NVIDIA GPU (which I don’t need anyway).

Ok, so how do I upgrade the firmware? fwupd comes to the rescue:

fwupd is an open source daemon for managing the installation of firmware updates on Linux-based systems, developed by GNOME maintainer Richard Hughes…” - Wikipedia.

I was a few keystrokes away from getting all my issues solved!
Dell put in a lot of effort to make sure fwupd works great with their products, So I wasn’t suprised that my laptop is supported.

$ fwupdmgr refresh
$ fwupdmgr update
No devices can be updated: Nothing to do

$ fwupdate --supported
Firmware updates are not supported on this machine.

What?! but why?! fwupdmgr recognizes my devices:

$ fwupdmgr get-devices
Intel AMT (unprovisioned)
...

XPS 15 9560 System Firmware
...

Integrated Webcam HD
...

GP107M [GeForce GTX 1050 Mobile]
...

So what’s wrong? I’m connected to AC, I’m running as root, I got UEFI Capsule Updates turned on.

Oh wait. I’m not using UEFI. No problem! let’s migrate!

https://docs.fedoraproject.org/f26/install-guide/install/Booting_the_Installation.html

My first thought: “Oh shit. I’m f\cked”*. My second thought: “that doesn’t make any sense!.

Game Plan

All I need is a simple grub-mkconfig while booted in UEFI mode, but how?

  1. Convert my paritition table to GUID Partition Table
  2. Free up some space for an EFI Partition /boot/efi paritition
  3. Update GRUB to use UEFI

Before we continue, I want to share we you my own partition table:

Disk /dev/sda: 953.9 GiB, 1024209543168 bytes, 2000409264 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos

Device Start End Sectors Size Type
/dev/sda1 2048 1953791 1951744 953M Linux filesystem
/dev/sda2 1953792 60549119 58595328 28G Linux swap
/dev/sda3 60549120 493574143 433025024 206.5G Linux filesystem
/dev/sda4 493574144 1669111807 1175537664 560.6G Linux filesystem
/dev/sda5 1669111808 2000203775 331091968 157.9G Linux filesystem

I have two OS’s installed. Arch & Fedora -

  • Arch: /boot is mounted at /dev/sda1 and / is mounted at /dev/sda3.
  • Fedora: / is mounted at /dev/sda4.

Both use /dev/sda2 for swap, and /dev/sda5 has some other data.
I don’t need Arch anymore, and would like to migrate Fedora to UEFI.

LiveUSB

I know that most of the changes I had to do couldn’t be done on mounted volumes, ,so I had to use a LiveCD. But nobody uses LiveCD’s nowadays - LiveUSB is the word on the streets.

I had two options. Either Download a LiveCD and burn it, or use Fedora Media Writer.

Booting into UEFI mode

First of all - I changed my BIOS configuration to boot up in UEFI mode.
Then, because I’m paranoid, I checked that I’m actually booted up in EFI mode:

$ sudo efibootmgr
EFI variables are not supported on this system.

Damn! that meant I wasn’t loaded into EFI, but I was ??

$ sudo modprobe efivarfs
$ sudo efibootmgr
BootCurrent: 0013
Timeout: 0 seconds
BootOrder: 0000
Boot0000 Fedora

Bam!

Convert parition table to GPT

I got the LiveUSB installed on a company thumb drive. Now I need to convert my paritition table from dos to GUID (GPT).

This step is rather simple. I Used gdisk:

# shouldn't require a password
$ su
$ gdisk /dev/your/device
# gdisk will now prompt that it wants to convert the partition table.
# press 'w' to save and you're done.

Free up space

I actually had another OS installed at the beginning of the partition table which I didn’t use anymore, so I just deleted it and recreated the EFI parition there.

If you don’t have one, install GParted and use it to free up ~500mb.

A few notes:

  • The EFI boot parition can be shared between OS’s. if you have one for Windows, no need to create another one.
  • The parition location isn’t important - it doesn’t have to reside in the beginning of the block for instance.
  • A parition size of ~500mb should suffice.

Update GRUB

Recap

I’ve got a new GPT partition table with an EFI partition at the beginning:

Disk /dev/sda: 953.9 GiB, 1024209543168 bytes, 2000409264 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt

Device Start End Sectors Size Type
/dev/sda6 2048 1953791 1951744 953M EFI System
/dev/sda7 1953792 60549119 58595328 28G Linux swap
/dev/sda4 60549120 1669111807 1175537664 413.1G Linux filesystem
/dev/sda5 1669111808 2000203775 331091968 157.9G Linux filesystem

chroot

I need Fedora to mount /boot/efi on boot, and configured to use UEFI. chroot to the rescue!

For those of you that have never heard of change root, Wikipedia provides a good explanation:

Chroot is an operation that changes the apparent root directory for the current running process and their children.

A program that is run in such a modified environment cannot access files and commands outside that environmental directory tree. This modified environment is called a chroot jail.

So back to where we were… Let’s chroot and get this over with.

# just login as root
$ sudo su
# mounting everything
$ mount /dev/sda4 /mnt/fedora
$ mount /dev/sda6 /mnt/fedora/boot/efi
$ mount -t proc proc /mnt/fedora/proc/
$ mount --rbind /sys /mnt/fedora/sys/
$ mount -t efivarfs efivarfs /sys/firmware/efi/efivars
$ mount --rbind /dev /mnt/fedora/dev/
$ mount --rbind /var /mnt/fedora/var/

# copying over the efi mount point
# you might want to comment-out any /boot mounts you might have
# this step is crucial, because we need an fstab entry for the efi partition.
$ grep "/boot/efi" /etc/fstab >> /mnt/fedora/etc/fstab

# chroot into your system
$ chroot /mnt/fedora /bin/bash

Awesome. I’m in my Fedora. Now I need to follow Fedora’s Updating GRUB 2 configuration on UEFI systems.

TL;DR:

Oh wait! My paranoid self keeps asking to check that I have an fstab entry
for the efi partition. Let’s give him some peace:

$ grep "/boot/efi" /etc/fstab

Good catch! I forgot to add it previously. Let’s add it:

$ echo "/dev/sda6 /boot/efi vfat defaults 0 1" >> /etc/fstab

TL;DR #2:

$ sudo dnf reinstall grub2-efi grub2-efi-modules shim
$ sudo grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg

It worked! grub2-mkconfig told me it found Fedora!

If it didn’t, I could’ve done this step manually:

$ sudo efibootmgr --create --disk /dev/sda --part 6 --loader /EFI/fedora/grubx64.efi --label "Fedora"

Checking that it all works

I know it sounds stupid, because the OS already booted, but why not?

$ ls /sys/firmware/efi/efivars | wc -l
81

$ efibootmgr
BootCurrent: 0007
Timeout: 0 seconds
BootOrder: 0007
Boot0000* Windows Boot Manager
Boot0006* Linux-Firmware-Updater \fwupx64.efi
Boot0007* Fedora

See? that wasn’t too hard!

Upgrading Firmware

After I did all that, I reran fwupd:

$ fwupdate --supported
Firmware updates are supported on this machine.

Yay!

$ fwupdmgr refresh
$ fwupdmgr update
...
$ reboot

Done. By the way, ALL the issues I previously had were gone after upgrading!

The Guts n’ Glory of Database Internals

A year ago Oren Eini (a.k.a @ayende) wrote a series dubbed “The Guts n’ Glory of Database Internals”.

Instead of just explaining how databases work, He incrementally builds a database from scratch. He goes over most database essentials, so once done you’ll be able to understand how databases actually work. If you haven’t taken any database courses at Uni, this is a must in my opinion.

The series is built around a “book keeping’ system. The problem? We need to keep track of users and how often they log into the system

He begins at persisting the data to a simple csv file, and then raises issues with this solution. The next part in the series addresses those issues and raises new ones: from selection time, concurrency, durability, logging and more.

Each part builds upon the previous and most parts take around 5 minutes to read.
The series is made up of 20 parts and takes ~ 1.5hrs to read from start to finish. Not so bad, right?

By the way, @ayende writes really well, so it’s also fun to read:

  1. Persisting information | 6 minutes
  2. Searching information and file format | 4 minutes
  3. The LSM option | 3 minutes
  4. B+ Tree | 11 minutes
  5. Seeing the forest for the trees | 4 minutes
  6. Managing Records | 7 minutes
  7. Managing Concurrency | 6 minutes
  8. Understanding durability with hard disks | 6 minutes
  9. Durability in the real world | 4 minutess
  10. Getting durable, faster | 3 minutes
  11. Writing to a data file | 3 minutes
  12. The enemy of thy database is… | 4 minutes
  13. The communication protocol | 4 minutes
  14. Backup, restore and the environment… | 4 minutes
  15. The curse of old age… | 5 minutes
  16. What the disk can do for you | 2 minutes
  17. What goes inside the transaction journal | 8 minutes
  18. Log shipping and point in time recovery | 3 minutes
  19. Merging transactions | 4 minutes
  20. Early lock release | 2 minutes

My take on Modern C++

C++ is a big language that has evolved tremendously since it’s inception way back in the 1980’s.

Throughout the years, many million lines of code have been written in the language and a big portion of that code is using legacy features that aren’t considered good practice anymore.

Replacing C++?

There were many attempts to replace the lanaguage. All of them failed as far as I know.

Some attempts were made to subset the language in order to get rid of code & language dept, which hurt speed and portability.

The most recent hype is around rust, which is a blazing fast, memory safe systems programming language. I see a promising future for rust, and I’m actually learning it myself. But like Bjarne said in his talk Writing Good C++14, it would take ~10 years for a good language to make it to the mainstream.

C++ is already here. We need to find a way for people to write good C++ now.

Subset of Superset

Simply sub-setting the language won’t work. This is backed by previous, failed, attempts. In order to maintain the speed of C++ we need the low-level / tricky / close-to-the-hardware / error-prone / expert-only features. Those feature are the building blocks of for higher-level facilitiese & libraries.

Bjarne talked about the subject at CPPCon a few years back:

Bjarne said we first need to create a superset of the langauge, then subset it in order to get rid of the crud. In order to do so, we need supporting facilities to make the transition: from guidelines on how to write modern C++, to libraries that enpasulate the usage of messy & dangerous things so most programmers won’t need to use them.

What is Modern C++

What is modern C++? Put simply, C++ that is based on C++1x and uses modern best practices.

To really grasp the essence of Modern C++, read the Core Guidelines. But nobody does that right?

Talks & Books

I really liked Bjarne’s Writing Good C++14, Neil MacIntosh’s The Guideline Support Library: One Year Later, Herb Sutter’s Writing Good C++ by Default & Modern C++: What You Need to Know.

I’ve also read parts of Effective Modern C++ by Scott Meyes and found it useful.

C++ Seasoning

I find Sean Parent‘s C++ Seasoning talk so good that I think you have to see it. I wrote about it in my previous post: C++ algorithm Series #0.

The talk takes a look at many of the new features in C++ and a couple of old features you may not have known about. With the goal of correctness in mind, Sean shows how to utilize these features to create simple, clear, and beautiful code.

TL;DR: No Raw Loops, No Raw Syntonization Primitives, No Raw Pointers.

Sean also gave another talk on the subject at Amazon’s A9 Programming Converstaions course.

Writing Modern C++

These are my do’s and don’ts regarding modern c++. There are also other things I do in order to make sure my project’s are written well:

  • Use CMake to build your project
  • Use memcheck to detect memory leaks
  • Run fuzzers to validate input
  • and more …

[!] Are you using a package manager? please let me know.

Follow Well-Known Guidelines

First, follow C++ Core Guidlines. You don’t need to actually read it, there are tools like Clang-Tidy that have the core guidelines pre-baked. Once you get a warning please go ahead and read the whole guideline. It’s important to understand Why the guideline exists.

Second, consider following well-known coding conventions and guidelines. On many occasions you can find tooling that help you follow guidelines created by big projects / corporations.

For instance, Clang-Format has pre-baked support for LLVM, Google, Chromium, Mozilla & WebKit. Clang-Tidy has pre-baked checks that follow Boost, Google, LLVM and more.

Use standard / popular libraries as much as possible. I use:

I try to use the [Standard Library as much as possible.

Compiler Flags

Turn on warnings, and preferebly warnings as errors. I usually turn on Wall and Werror. They are anoyying, but a neccessery evil IMO.

RAII

A few days ago I watched a talk called “Modernizing Legacy C++ Code” where they suggested to use RAII everywhere:

I’m not suprised, I’m a huge fan of RAII. Not only it makes code cleaner, thus reducing bugs and memory leaks, it also has an extremely low performance impact (compared to golang’s defer, which is used for the same purpose)

If Your’e interfacing C code, consider creating a scope guard. I use my home-baked defer-clone for that purpose.

Const-Qualify Everything

On “Modernizing Legacy C++ Code” they also talked about using const everywhere. At first it sounded weird, but it actually made a lot of sense once they showed a few examples:

This is a rolling release. That is, I’ll keep updating this post with new insights.

C++ <algorithm> Series #0

The more I learn about an eco-system, the more I understand how little I know about it.
I’m a master of nothing. There’s always something new to know, and the list only grows.

This time around it’s C++ that haunts me.

I recently “binge-watched” a few CPPCon talks:

They were awesome, but one talk really got my attention - C++ Seasoning by Sean Parent.

I have this “I don’t know anything about programming” feeling every time I watch a Rich Hickey talk.
Sean’s C++ Seasoning (then Programming Converstaions #1 & #2) made me feel the same.
His talk reminded me, again, that I have a lot to learn.

One of the key points of his talk is that developers need to be familiar with the algorithm library,
and be able to extend it. During his talk, He magically transformed a few-dozen-lines of complex code into two, using <algorithm>.

I’m used to seeing python code refactoring that turn huge pieces of code into a few, which are easy to read and reason about. But in C++? wow.

Anyway, one of my takeaways is that I don’t use <algorithm> enough. It’s time to change that.

The <algorithm> series

I’ll go through every algorithm in [ C++’s stl, Adobe’s asl, Google’s abseil & Boost ] and will provide examples for each. I might even throw in some Introduction to Algorithms references.

This is a big project which will take a very (very) long time to finish, but it’s worth it.

Stay tuned.

Everything that is wrong with sudo, and how I'm planning to fix it

suex
suex

The last few weeks have been a bit crazy.

I found myself re-implementing gosu because it lacked features I needed: sick of sudoers NOPASSWD?.

Then, Even when I was done, I felt something was missing. runas, the tool I wrote, was half-way into becoming a sudo replacement and it bugged me I stopped mid way.

I looked around the web and found an amazing project called doas that is basically runas on steroids.

What is doas

doas is a utility that is aimed to replace sudo for most ordinary use cases.
Ted Unagst’s, an OpenBSD developer, explained why He originally wrote it in his blog post: doas - dedicated openbsd application subexecutor.

The gist is that sudo is hard to configure and does a lot more then the standard user needs. doas was created in order to replace sudo for regular folks like me and you.
Moreover, sudo lacks ‘blacklist’ behaviour which is extremely useful at times.

doas is relatively easy to configure, and an absolute joy compared to sudo. It’s also powerful enough for most daily use-cases.
IMO, the permit / deny concept of doas is so powerful that it’s enough to make the switch.

Implementing doas from scratch

The problem was that doas was written for OpenBSD.
I’m not running OpenBSD, so I looked around for a port.

All ports I found were half baked and poorly written.

Then I looked at the original source code, and decided I’m not going to port it.

Why? because it’s written in C, and I really don’t want to maintain C code.
Furthermore, the original code base lacked feature I introduced in runas which I really loved.

Instead, I decided to start this project. A complete re-implementation of doas.

This is my first attempt at writing a production quality, open source project from scratch.

I’m not there yet, but I’m determined on pushing this project into the main repositories of both ubuntu and fedora. More work has to be done in order to get there. for instance: Adding system tests & Getting the code audited.

Feel free to reach out if you want to contribute!

Project Goals

  • Secure. User’s shouldn’t be able to abuse the utility, and it should protect the user from making stupid mistakes.

  • Easy. The utility should be easy to audit, to maintain, to extend and to contribute to.

  • Friendly. Rule creation should be straight forward. Rule should be easy to understand and easy to debug.

  • Powerful. Rules should be short, concise and allow find-grained control.

  • Feature Parity. This project should have complete feature parity with the original utility.

To achieve these goals, the following design decisions were made:

  1. The whole project was implemented in modern C++.
  2. Explicit is better then implicit (for instance, rule commands must be absolute paths)
  3. Prefer using the standard library when possible - for the sake of security and maintainability.
  4. Commands are globs, which allows to use the same rule for many executables.
  5. Arguments are PCRE-compliant regular expressions, which allows to create fine-grained rules.

Getting started

You can find pre-compiled .deb and .rpm packages in the project’s GitHub Releases Page.

[!] Ubuntu PPA & Fedora Copr are coming soon.

You can also build from source. more information found at odedlaz/suex.

Changes compared to the original

Security checks

doas doesn’t check the owners & permissions of the binary and configuration file.
sudo checks those, but only warns the user.

This version ensures the binary and configuration file are owned by root:root.
It also ensures the binary has setuid, and that the configuration file has only read permissions.

Furthermore, only full paths of commands are allowed in the configuration file.
The idea is that privileged users (i.e: members of the wheel group) need to explicitly set the rule instead of depending on the running user’s path.

Edit mode

suex -E

suex allows any privileged user (i.e: members of the wheel group) to edit the configuration file safely.
Furthermore, if the configuration file is corrupted, privileged users can still access it and edit it.

The edit option is similar to visudo, it creates a copy of the configuration and updates the real configuration only when the copy is valid.

Non-privileged users are not allowed to edit the configuration.

Verbose mode

suex -V

suex allows to show logging information to privileged users. That information shows which rules are being loaded & how they are processed.

Non-privileged users are not allowed to turn on verbose mode.

Dump mode

suex -D

suex allows the user to dump the permissions it loaded to screen.
group permissions and command globs are expanded into individual rules as well.

privileged users see the permissions of all users instead of only their own.

Examples

Ted Unagst’s wrote a great blog post called doas mastery. Because the project has complete feature parity with the OpenBSD version, the mentioned post should be a good starting point.

Never the less, there are some powerful enhancments in this release that deserve special attention.

fine-grained package management

deny odedlaz as root cmd /usr/bin/dnf args (autoremove|update|upgrade).+
permit keepenv nopass odedlaz as root cmd /usr/bin/dnf args (autoremove|update|upgrade)$

The first rule denies odedlaz of running dnf as root with any arguments that start with autoremove, update & upgrade and have other arguments as well.

The second rule allows odedlaz to run dnf as root only with autoremove, update, upgrade and no other arguments.

These protect odedlaz from from accidentally running dnf autoremove -y or dnf upgrade -y, even if He’s a privileged user (a member of the wheel group).

On the other hand, it allows odedlaz to run these commands without a password (nopass) if they are executed without any trailing arguments.

rm -rf protection

deny odedlaz as root cmd /bin/rm args .*\s+/$

The above rule protects odedlaz from accidentally running rm -rf / and the like.

one rule, multiple executables

permit keepenv nopass odedlaz as root cmd /home/odedlaz/Development/suex/tools/* args .*

The above rule allows odedlaz to run any executable found at /home/odedlaz/Development/suex/tools with any arguments, as root without requiring a password.