|
| News | Profile | Code | Photography | Looking Glass | Projects | System Statistics | Uncategorized |
Blog |
From the RPi annoyances department..
I did a kernel upgrade on one of my Raspberry Pis earlier today by installing the latest raspberrypi-kernel from testing (1.20170515-1). It installed Linux 4.9.28+ and I instantly realized there was something wrong since the Wi-Fi was very latent and would periodically disconnect from any nearby WAPs. I use the Edimax EW-7811u USB stick since this Raspberry Pi is one of the original model Bs without Wi-Fi.
After debugging a bit and searching the Internet it appeared that I was now using the wrong driver, rtl8192cu instead of 8192cu:
(henry:22:18:PDT)% uname -a Linux henry 4.9.28+ #998 Mon May 15 16:50:35 BST 2017 armv6l GNU/Linux (henry:22:19:PDT)% lsmod|grep -i rtl rtl8192cu 79230 0 rtl_usb 12234 1 rtl8192cu rtl8192c_common 57894 1 rtl8192cu rtlwifi 85339 3 rtl_usb,rtl8192c_common,rtl8192cu mac80211 650547 3 rtl_usb,rtlwifi,rtl8192cu cfg80211 525710 2 mac80211,rtlwifi
Apparently, before this upgrade the 8192cu driver was being used. So, I added "blacklist rtl8192cu" to /etc/modprobe.d/raspi-blacklist.conf and rebooted. Bingo, the right module is now being used:
(henry:23:04:PDT)% lsmod|grep 8192 8192cu 597868 0 cfg80211 525710 1 8192cu
No more Wi-Fi drops!
It's been awhile since I've posted something here, again.
Devon and I recently completed a Caribbean cruise on the Norwegian Escape. The ship itself was fairly new (built in 2015) but was packet to the gills. There were over 4,100 passengers! As a result, it was difficult to use the elevators from time to time.
We departed out of Miami and visited Nassau, St. Thomas, and Tortola. I uploaded some random photos. The image above is from the north side of Blue Lagoon, which was pretty deserted.
I started messing with BIRD the other day to work around some IPv6 issues with Quagga. The configuration is fairly simple, but I ran into a weird issue where it picks the wrong interface IPv4 address from some of my OpenVPN tunnels. For example, I've got these interfaces:
(storm:0:06:EST)% ip a s tun0
5: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1456 qdisc pfifo_fast state UNKNOWN group default qlen 100
link/none
inet 10.3.254.44 peer 10.3.254.43/32 scope global tun0
valid_lft forever preferred_lft forever
(storm:0:06:EST)% ip a s tun1
4: tun1: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1456 qdisc pfifo_fast state UNKNOWN group default qlen 100
link/none
inet 10.3.254.81 peer 10.3.254.80/32 scope global tun1
valid_lft forever preferred_lft forever
Here's what BIRD sees:
bird> show route protocol direct1 10.3.4.64/32 dev lo [direct1 23:53:06] * (240) 10.3.254.43/32 dev tun0 [direct1 23:53:06] * (240) 10.3.254.80/32 dev tun1 [direct1 23:53:06] * (240) 192.168.150.0/24 dev eth0 [direct1 23:53:06] * (240)
The addresses shown are the remote end of the OpenVPN tunnels, not the local end, which I'd expect.
Why?
Update: Well, this should have been obvious:
(storm:0:34:EST)% ip r s p kernel 10.3.254.43 dev tun0 scope link src 10.3.254.44 10.3.254.80 dev tun1 scope link src 10.3.254.81 192.168.150.0/24 dev eth0 scope link src 192.168.150.105
I suppose I'll have to figure out how to get the link source to be visible to BIRD so I can advertise it, which is one of my odd requirements here.
As I was doing upgrades on a few of my Raspberry Pi machines (running Raspbian), I ran into a situation where apt-get would SIGABRT on any action. Depending on the shell, the message to the user is a little different and also overwrites a different amount of APT's status message. Here's Zsh:
(storm:20:50:EST)% sudo apt-get --purge remove quagga-pimd quagga-isisd quagga-ospfd quagga-ospf6d zsh: abort sudo apt-get --purge remove quagga-pimd quagga-isisd quagga-ospfd
Here's Bash:
prox@storm:~$ sudo apt-get --purge remove quagga-pimd Aborted package lists... 48%
No matter what the operation (install, upgrade, update, remove), APT would just get a SIGABRT. There was nothing in the kernel log buffer and the RPi 3 I was on at the time wasn't out of disk space or memory. Since this RPi had just completed a dist-upgrade and I had rebooted it, everything was more-or-less up-to-date with the "testing" channel of Raspbian. I searched the web and couldn't find anything related to "abort" or "SIGABRT" on Debian or Raspbian. There were also no bugs filed against any of the relevant packages. So, I figured that I had just hit a bug and started downgrading. I used dpkg-repack on another RPi 3 of the following packages, which I downgraded to:
apt_1.3~rc4_armhf.deb
apt-utils_1.3~rc4_armhf.deb
dpkg_1.18.10_armhf.deb
dpkg-dev_1.18.10_all.deb
libapt-inst2.0_1.3~rc4_armhf.deb
libapt-pkg5.0_1.3~rc4_armhf.deb
libc6_2.23-4_armhf.deb
libc6-dbg_2.23-4_armhf.deb
libc6-dev_2.23-4_armhf.deb
libc-bin_2.23-4_armhf.deb
libc-dev-bin_2.23-4_armhf.deb
libc-l10n_2.23-4_all.deb
libdpkg-perl_1.18.10_all.deb
locales_2.23-4_all.deb
multiarch-support_2.23-4_armhf.deb
On the upgraded RPi, I'd been running APT 1.4~beta4, libc 2.24-9, and dpkg 1.18.18. Unfortunately, none of the downgrades did the trick. My next step was to downgrade the kernel, but I started to think something was maybe gummed up with APT itself, possibly in its database since it always received the SIGABRT about 50% of the way through reading package lists. I ended up looking up the various storage directories and decided to clear out /var/lib/apt/lists. I did another apt-get upgrade and things started working again:
(storm:21:55:EST)% sudo apt-get update Get:1 http://archive.raspberrypi.org/debian jessie InRelease [22.9 kB] Get:2 http://mirrordirector.raspbian.org/raspbian testing InRelease [15.0 kB] Get:3 http://archive.raspberrypi.org/debian jessie/main armhf Packages [141 kB] Get:4 http://mirrordirector.raspbian.org/raspbian testing/main armhf Packages [11.7 MB] Get:5 http://archive.raspberrypi.org/debian jessie/ui armhf Packages [53.6 kB] Get:6 http://mirrordirector.raspbian.org/raspbian testing/contrib armhf Packages [56.0 kB] Get:7 http://mirrordirector.raspbian.org/raspbian testing/non-free armhf Packages [95.1 kB] Get:8 http://mirrordirector.raspbian.org/raspbian testing/rpi armhf Packages [1,360 B] Fetched 12.1 MB in 24s (493 kB/s) Reading package lists... Done (storm:21:59:EST)% sudo apt-get --purge remove quagga-pimd quagga-isisd quagga-ospfd quagga-ospf6d Reading package lists... Done Building dependency tree Reading state information... Done The following packages were automatically installed and are no longer required: libquagga0 quagga-bgpd quagga-core quagga-ripd quagga-ripngd Use 'sudo apt autoremove' to remove them. The following packages will be REMOVED: quagga* quagga-isisd* quagga-ospf6d* quagga-ospfd* quagga-pimd* 0 upgraded, 0 newly installed, 5 to remove and 8 not upgraded. After this operation, 957 kB disk space will be freed. Do you want to continue? [Y/n]
Apparently something had gotten corrupted in the downloaded package lists and APT didn't handle it properly. I'd submit a bug for this but I didn't think to backup /var/lib/apt/lists beforehand so I'm not sure how to reproduce this. Anyway, if you get a SIGABRT when running apt-get, rm /var/lib/apt/lists/* and things will be happy again.
Also, in other news, Lady Gaga's Super Bowl LI performance was pretty decent. The drones were a nice touch.
I haven't blogged in awhile.. so here are some updates.
I recently moved dax, which is the FreeBSD-based host that powers this web server, to a VM and away from Internap's Agile service. dax used to be a dedicated server hosted by Voxel dot net and provided IPv6 connectivity to the rest of my network since September of 2005. However, things started going downhill when Internap acquired them in 2012. Technical support turned into a horror story and they recently indicated to me (in an IPv6 support ticket from hell) that the Agile "legacy" services were going to be done away with later in 2016 (not sure if this is true).
I migrated all of my sites off the IPv6 PA /56 I had from Internap and onto my own PI /44. I moved dax to a VM on excalibur, a dedicated server hosted by Choopa, LLC. I'm now Internap-free and running everything off of AS395460.
Anyway, excalibur's got an Intel Xeon E3-1240 v2 CPU with 4x cores and should run at 3.40 GHz. Normally, with servers of mine that run VMs, I instruct cpufreq-set(1) to run all cores with the performance governor, which is supposed to direct all cores to run at their maximum clock rate. However, with the E3 Xeon CPUs, this doesn't work. The clock frequencies of each oscillate based on load and cause a considerable latency penalty, which is fairly visible in packet forwarding (causes jitter). Here's a snapshot:
(excalibur:2:12:EDT)% cat /proc/cpuinfo|grep -i MHz cpu MHz : 1714.742 cpu MHz : 2652.000 cpu MHz : 3563.492 cpu MHz : 3799.898 cpu MHz : 3722.070 cpu MHz : 2126.195 cpu MHz : 3800.164 cpu MHz : 2313.062
Searching on the web indicated I would need to completely disable the C-states in the BIOS, something I wasn't really willing to do and didn't feel like the correct solution. I then came across another post that indicated I should write "0" to /dev/cpu_dma_latency but keep the file open. So, I did this:
(excalibur:2:12:EDT)# cat > /dev/cpu_dma_latency 0
.. and didn't hit ^D. Sure enough:
(excalibur:2:12:EDT)% cat /proc/cpuinfo|grep -i MHz cpu MHz : 3599.882 cpu MHz : 3599.882 cpu MHz : 3599.882 cpu MHz : 3599.882 cpu MHz : 3599.882 cpu MHz : 3599.882 cpu MHz : 3599.882 cpu MHz : 3599.882
Seriously? There's a whole nice structure in /sys/devices/system/cpu/cpufreq/* that Linux has used for over a decade to control CPU frequency scaling and we've now a one-off /dev character device that controls such things on a modern CPU?
Well, considering the stupidity of systemd that's supposedly accepted by most Linux distributions now, I guess I shouldn't be surprised that this type of hacky interface exists. At least I've got a way to emulate the behavior of the performance governor without mucking with BIOS settings. It turns out this also works on my E3 1245 v3 Xeon I've at home in vega, too.
Update: This is a bad idea. This causes all CPUs to run very hot even when utilization is low, which confuses me:
Update: The right solution to this problem is to just disable the Intel P-states driver by passing intel_pstate=disable on the kernel command line. The acpi-cpufreq driver is used instead and operates the way it should. See comments for more information.
Almost as a continuation of Dead RPis and Interface Naming, I've now encountered some systemd/udev-related problems when creating VLAN interfaces on Linux (Debian) on one of my routers after upgrading to bleeding-edge stuff (Linux 4.6.0 and udev/systemd 230):
Say I have one of the following in /etc/network/interfaces:
auto eth1.35
iface eth1.35 inet static
address 10.3.7.250
netmask 255.255.255.252
or..
auto vlan35
iface vlan35 inet static
address 10.3.7.250
netmask 255.255.255.252
vlan-raw-device eth1
I'll get this:
(starfire:22:23:PDT)# ifup vlan35 Set name-type for VLAN subsystem. Should be visible in /proc/net/vlan/config Added VLAN with VID == 35 to IF -:eth1:- (starfire:22:24:PDT)# ip link show dev vlan35 Device "vlan35" does not exist. (starfire:22:24:PDT)# ip link|grep rename 61: rename61@eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 (starfire:22:24:PDT)#
Obviously, I expected vlan35 to be created, but udev renamed it immediately to rename61. The only udev rules that are acting on this, from what I can tell, are /etc/udev/rules.d/76-netnames.rules and /lib/udev/rules.d/19-ifrename.rules:
(starfire:22:27:PDT)# cat /lib/udev/rules.d/19-ifrename.rules
# udev rules to properly integrate ifrename.
# Renaming is done using /etc/iftab, with full ifrename functionality.
# Require udev version 107 or later.
# Please double check the path to ifrename, and make sure its available
# when udev runs (i.e. on boot partition).
# Enable this rule to test with udevtest.
#ENV{UDEV_LOG}=="6", SUBSYSTEM=="net", ACTION=="add", IMPORT{program}="/sbin/ifrename -D -V -u -i %k", NAME:="$env{INTERFACE}"
# Main ifrename rule.
# If interface is found in /etc/iftab, subsequent rename rules are bypassed.
# If interface is not found in /etc/iftab, subsequent rename rules applies.
SUBSYSTEM=="net", ACTION=="add", IMPORT{program}="/sbin/ifrename -u -i %k", NAME:="$env{INTERFACE}"
(starfire:22:27:PDT)# cat /etc/udev/rules.d/76-netnames.rules
# Generated by netnames.sh on Sat Jul 9 19:09:59 PDT 2016
SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="00:1b:21:08:2b:0a", NAME="eth0"
SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="00:04:23:5f:4c:d9", NAME="eth1"
SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="00:1b:21:08:2b:0b", NAME="eth2"
SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="00:04:23:5f:4c:d8", NAME="eth3"
SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="00:08:74:c5:d4:61", NAME="eth4"
(starfire:22:27:PDT)#
The former comes with the udev package but the latter was created by me since I needed all of the NICs in my router to have consistent and traditional names. I'd add these VLAN interfaces to 76-netnames.rules but there's nothing unique I can match since the MAC addresses on the VLAN interfaces are copied from the physical interface.
It's pretty obvious that udev is renaming VLAN interfaces and getting in the way. I should normally be able to hack around this by a pre-up or up directive in /etc/network/interfaces but the renameXX interfaces are not consistent (but predictable:
