Event site: https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/
Today as I am writing this, it is New Year’s Eve, and I am reflecting on 2022 and thinking ahead to 2023. It seems like a perfect time to collect my notes from this past KubeCon into a succinct post with a particular focus on what topics I think will be important in 2023.
This was my first ever KubeCon, and it was a fantastic experience. I learned a lot about a wide variety of topics. This quick post certainly will not even attempt to capture the entire experience. If you haven’t ever attended, and you get an opportunity to attend, either in-person or virtually, I highly recommend it.
This year, I attended virtually. I wrote some thoughts about virtual vs in-person attendance in my post on AWS re:Invent 2022. While the specifics differ between the two conferences, the broad strokes should remain relevant, so if you are interested in that topic, I’d suggest giving that post a look.
Time to make predictions! It will be fun to look back in a year’s time to see if any of these predictions were even close to being accurate.
Without further ado, I believe these will be the top three most important or influential trends or topics in 2023 in the Kubernetes space:
For each of these topics I am including one or two presentations and some supporting links for further reading.
Docker has announced beta support for WASM Containers. I feel that this will lower the barrier to entry significantly and allow many developers to experiment and see if this technology can help their technology and business goals.
There are two WASM-related presentations from KubeCon which I caught which I think are worth a look.
I think the presentation offers a decent overview of WebAssembly.
Specifically, the WasmEdge runtime is presented.
This presentation approaches WASM from a very different angle, which I found very intriguing: distributing code updates as a configuration change. More specifically, using WebAssembly to distribute a single function as a WASM binary which is imported into the main firmware, e.g. for an IoT Edge device which is otherwise difficult to update.
This is a great talk. Quick and to the point, it explains what works currently, what doesn’t, and goes into some of the details of the challenges of implementing support for User Namespaces in K8s - KEP-127.
To help convince you that this will be an important topic in 2023, take note of the demo in the presentation, which shows a container-escape-to-root exploit which is mitigated by this new feature, even as the underlying host remains unpatched against the CVE used in the demo.
There are currently some limitations, e.g.: no stateful pods, but these are likely to be addressed soon. Requires K8s v1.25.0+
In short, Gateway API seeks to create a unified API spec for both ingresses and service-meshes across various implementations.
I am excited about this since it should hopefully create greater cohesion and maturity in two related areas where Kubernetes seems to be more fragmented and confusing than it needs to be.
]]>I just thought I’d share some of my notes regarding sessions which I found interesting in case it is helpful for anyone.
I had been hoping to get this post out much closer to the end of the event, but life got busy as it often does. At least the post is making it out within the same month!
This was actually my first re:Invent, so I can’t make any comparisons to previous re:Invents. Also, I attended the event virtually since there were various reasons I was unable to travel this year. I can offer some thoughts on attending virtually, but cannot compare it directly to attending in-person.
Unless I was doing something wrong (I very much doubt it, but it is certainly possible), the only sessions which were available live via the virtual attendee portal were keynotes and leadership sessions. Then, after some time, the other sessions were posted to an official AWS Events YouTube channel. On that channel, the breakout sessions are helpfully organized into playlists by topic.
I don’t recall exactly how long it would take before a breakout session would become available on YouTube, but I feel it was anywhere from 2 hours to the next day.
The virtual attendance experience is very different from in-person. Of course, both have their pros and cons.
I got some sense of the difference since many of my co-workers attended in-person and there was a Slack channel dedicated to re:Invent. Via the Slack channel I got to see photos of many co-workers meeting for the first time - either due to remote-work or from being on geographically dispersed teams. I certainly regret that I was unable to attend in-person this year to take part in those experiences. Chalk this up to the “cons” column.
On the other hand, folks who attended in-person had to carefully plan their day to get into the sessions which most interested them. If they happened to have interests which span multiple tracks, they might have found themselves needing to take a shuttle from one hotel to another and to factor in this travel-time. With virtual attendance, I was able to avoid all travel-time, and watch any break-out from any track as soon as it became available on YouTube.
I also saw some reports of sessions becoming fully booked - not a problem for virtual attendees.
Watching the sessions on YouTube also afforded the opportunity to pause to study diagrams and read slides, and of course to rewind if needed. Another great benefit is the ability to increase the playback speed for slow speakers.
I also attended KubeCon this year (also virtually). I had felt certain that AWS would have more resources and would therefore have a far greater selection of sessions being streamed live. I was initially very disappointed to find that KubeCon actually had a greater selection of sessions available live, or at least same-day.
Whatever shortcomings and trade-offs I experienced by attending virtually, I’m still grateful to have been able to learn and participate.
I think probably the best approach would be to attend in-person and to plan to catch-up on the talks which were full or that you couldn’t make it to via the YouTube channel later.
Just a small subset of announcements that I personally thought were interesting.
Amazon CloudWatch Internet Monitor Preview
Just a small subset of sessions which I personally thought were interesting and worth recommending.
Really excellent talk especially for those who are newer to AWS Serverless. It does a great job of modeling organic growth of an application from MVP to an app with over 100 active subscribers and how the app can be extended with new features with minimal refactoring, while still starting small and simple.
Very interesting talk where they did a deep and nuanced comparison between Rust and Java across multiple dimensions (memory consumption, latency, throughput, energy consumption, cost).
There were some expected results, but many of the findings were a bit surprising. Number 5 will shock you! :-)
For example, when they iterated on the Java app to use GraalVM, and the Rust app was using Tokio async/await, the Java app actually out-performed the Rust app in terms of execution speed. Then they refactored the Rust app to use a thread-pool instead of Tokio async/await and surpassed the Java performance.
If you find this topic interesting and like to get in the weeds, you will enjoy this talk.
https://github.com/aws-samples/serverless-rust-demo
https://github.com/aws-samples/serverless-graalvm-demo
Starts with a Matrix skit, which is worth watching on its own. The skit shows what a fully synchronous world would look like, versus the async world we actually live in. Async is used as the connective tissue which binds the keynote together.
Good overview/review of some basic container concepts.
Covers Finch.
Container building tips / tricks at the end. Good for any teams getting started with containers.
I didn’t take notes for all the sessions I watched, but if I did have any notes I’m dropping them here.
Very clear session on multi-region design patterns.
GraphQL pub/sub. Good, clear explanation of the topic and nice comprehensive demo.
Details and demo of a new feature: “Amazon ECS Service Connect”.
If you are intersted in the Karpenter project, but not familiar with it this is a good overview of what Karpenter does and how it works.
]]>I am by no means a Rust expert, but I’ve found these resources helpful and I hope some of them will help you too!
As I come across new resources, I’ll be sure to update this page.
The first place you should start is the official docs - they are fantastic!
Particularly, I think the The Rust Book should be everybody’s first stop.
It is official, and it is entirely free. I’ve read it cover-to-cover and re-read certain sections. I love that the example programs in the book are simple enough to easily understand, but still complex enough to seem like they are modeling a real-world problem that is worth solving.
If you come from another compiled language or you enjoy more to-the-point and terse examples, you may enjoy Rust By Example.
It is also an excellent resource after you have read The Rust Book and just need a quick review of a specific topic.
The Rust Book and Rust By Example are not the only official Rust learning resources. More can be found here: https://www.rust-lang.org/learn.
The No Boilerplate YouTube channel has some interesting lightning talks about various Rust topics.
You won’t be learning any topics in-depth in the seven to twelve minute videos, but they often offer an interesting angle on a topic. Additionally, the videos might serve as a nice bit of motivation to continue your Rust studies, as you get glimpses of things that are possible in Rust, and some of Rust’s unique value propositions.
If you only watch one video, watch: Rust: Your code can be PERFECT
Sure, the title has a strong whiff of clickbait and the video has more than a strong taste of Rust Kool-Aid, but, in my mind, the point is still valid: with Rust, once your code compiles, you’ll likely spend more time implementing new features and fixing logical bugs, than fixing bugs arising from safety issues or stability issues.
There are some really excellent Rust posts on https://cliffle.com/blog.
Some posts of note:
Highly recommended reading. The Typestate Pattern is not unique to Rust, but Rust’s language design lends itself very well to this powerful pattern.
Why Rust mutexes look like they do
I love this type of article. It is in the format where a common complaint is explored and, in good faith, the author attempts to address the complaint. In this case, the author explores an alternate mutex implementation in Rust with the goal of it being more like a C/C++ mutex. As we go on this journey with the author, we find several major problems. In order to solve those problems, we find that the only viable solution ends up looking almost exactly like the Rust standard library mutex implementation.
A great reason to read this article is that you might learn something about C and C++, mutexes, and software engineering in general.
Prefer Rust to C/C++ for new code
If you are looking for an article to help convince your boss (or yourself) to take the plunge and invest the time and effort to switch to Rust, this article might be for you. As an added bonus, this article is extremely pragmatic and does not blindly recommend Rust - it includes an excellent “When to not use Rust” section as well.
Amos, over at https://fasterthanli.me/ writes intensely technical, excellent, and almost-always Rust-centric posts.
The blog features Cool Bear, a cartoon bear who pops into the margins to ask insightful questions and sometimes poke and prod Amos into re-writing a solution in a more elegant way. Cool Bear is very cool.
Some posts of note:
An infamous post, with an appeal to choose Rust over Go. Although it may have started flame-wars, the post is very sincere and his argument is very carefully laid out. Very strongly recommended reading.
A sequel to I want off Mr. Golang’s Wild Ride. If you made it through the original, you will definitely want to read this one.
All the posts in the Advent of Code series offer some great insights into solving various problems with Rust. It is interesting to see which crates he selected for the various problems. I recommend starting at Day 1 and working your way forward.
Learn Rust With Entirely Too Many Linked Lists
I’ve not read through the entire book, but it seems promising. I would probably not recommend this as your first foray into Rust-land - The Rust Book is a much kinder, gentler, and probably more useful introduction.
One of the common arguments against using Rust goes something like: “It is impossible to implement a linked list in (safe) Rust, so you shouldn’t use Rust”. The book offers a very thorough exploration of this topic.
As advertised, it definitely involves too many linked lists.
]]>When I originally wrote this post back in early 2021, I believe Bitwarden (the company) was providing generic .deb and .rpm packages for Bitwarden (the desktop application) and an AppImage pakage. If memory serves, at that time they were not publishing the app as either snap or Flatpak packages. Additionally, the AppImage package was self-updating, unlike the .deb / .rpm packages.
As of 2026, I would reccomend skipping the AppImage installation method and either installing via flatpak or snap, since both are available.
https://flathub.org/en/apps/com.bitwarden.desktop
https://snapcraft.io/bitwarden
The original AppImage and rpm and deb packages continue to be available, which is nice.
If you really want to proceed with the AppImage installation method, then maybe the original 2021 post, below, could still have some helpful hints for you.
I’ve been wanting to try the password manager Bitwarden for some time now. The recently announced changes to the free edition of LastPass was just the push I needed to give Bitwarden an earnest try.
I can report that the process of exporting my data from the LastPass browser extension to csv, then importing it into Bitwarden was easy and painless. While the steps were self-explanatory, I do appreciate that Bitwarden has a reference document available. To be clear, I exported from the LastPass browser extension and then imported into the Bitwarden web interface.
I really appreciate that while Bitwarden provides the usual iOS, Android, Windows, and macOS binaries, they also provide a cross-distro compatible Linux package in the form of an AppImage. While the Snap vs Flatpak debate seems to get the most attention, it is nice to see AppImage getting some love.
Disclaimer: this is not the only way to handle installation, and may not be the best method, depending on your needs, but this is how I like to do it.
This method installs the AppImage binary into your home directory, so we assume that no other users on your system will need access to Bitwarden.
# Create a bin directory in your home:
mkdir ~/bin
cd ~/bin
# Get the latest available AppImage Bitwarden package:
wget "https://vault.bitwarden.com/download/?app=desktop&platform=linux" -O Bitwarden.AppImage
# Make the AppImage executable:
chmod 775 Bitwarden*.AppImage
# Since the Bitwarden AppImage binary will change its name whenever it
# it auto-updates, I like to create a wrapper-script to call it:
cat << EOF > ./bitwarden.sh
#!/bin/bash
$HOME/bin/Bitwarden*.AppImage &> $HOME/bin/bitwarden.log &
EOF
# Don't forget to make the wrapper script executable:
chmod 770 bitwarden.shNext, I like to integrate the application into my KDE menu. This should work regardless of the underlying distro, as long as your desktop environment is KDE (and why wouldn’t it be?).
Right-click on the KDE menu, click “New Item”, type the name “Bitwarden”, and click “ok”. On the left hand side, drag the Bitwarden menu item into the category that makes the most sense to you - I usually put it in the “Office” category. On the right-hand side: Optionally add a description and comment, but definitely modify the “Command” field to: /bin/bash -c "/home/<YOUR USER NAME>/bin/bitwarden.sh". You can also add an icon by clicking the empty square in the KDE Menu Editor (right-hand side). Usually I search within system icons for “lock” and just use a generic padlock icon. When done, click “save” to close out of the Menu Editor. Note: if you do not set a custom icon, the menu entry will just use a default “empty page” icon - but when it is running in the task manager you will actually see the correct Bitwarden logo.
I have heard there is an AppImage daemon appimaged as well as a Launcher project AppImageLauncher, either of which would probably help with the automation of integrating AppImage apps into GUI menus, and also might add some automation around updating the AppImages themselves. I should probably look into these if I start using AppImages on a larger scale. If I do, I’ll try to come back and edit this post.
It doesn’t much matter where you put the AppImage and whether or not you integrate it into the menu of your desktop environment of choice. As far as I can tell, Bitwarden will store local data in your home directory, as a well-behaved desktop application should, in ~/.config/Bitwarden.
Finally, just a friendly reminder to ensure you have configured two factor authentication.
Happy password managing!
]]>In case anyone finds it helpful, I’m sharing my playbook for patching RHEL-5 and RHEL-7 hosts.
In an ideal world there would be no more RHEL-5 hosts since even the Extended Life Cycle Support for RHEL-5 ended on November 30, 2020. However, I think we can agreee that we don’t live in an ideal world, unfortunately.
The playbook assumes that RHEL-7 hosts can reach thier repos normally, but that the RHEL-5 hosts are on a restricted vlan without internet access, and also that the Ansible control host can reach the internet to fetch packages and push said packages to the RHEL-5 hosts.
Running the playbook will download the RHEL-5 32-bit & 64-bit rpm packages and PGP keys from the sudo-project to the Ansible host. This is idempotent, so you only need to fetch the files once. Note: you should add a line like *.rpm to your .gitignore to avoid checking rpm packages into git.
Next, we have two blocks, each with a when: conditional. The first block executes when the OS is RHEL-7, based on Ansible facts and simply uses the yum module to ensure the sudo package is at the latest version.
The second block similarly only runs when the host is RHEL-5. It copies the appropriate RPM (based on the system architecture) to the remote host along with the PGP keys. Next, it imports the pgp keys, then does a local install of the rpm package, and finally removes all temp files which were copied over.
If you need to patch RHEL-6 hosts, the playbook could easily be extended with a similar pattern.
]]>I wrote a quick Ansible playbook to check if a host is vulnerable or patched.
You can run the playbook like this:
anisble-playbook -K \
-i test-inventory \
check-sudo-vuln-CVE-2021-3156.yml \
-e target=testingThe -K is to “Prompt for the password to use with sudo, if any”, if you have passwordless sudo for your account on your remote servers, you can omit this flag.
The -i is to allow you to point to a particular Anisble inventory file. I’m using one called “test-inventory” for purposes of … testing.
The -e is to pass an extra variable where we define the variable “target” as being equal to “testing”. The “testing” target is a host group defined in the inventory file.
The simple test-inventory file looks like this:
[testing]
host01
host02Running the playbook will produce output similar to:
TASK [Show Vulnerable Hosts] ********************************
skipping: [host01]
ok: [host02] => {
"msg": "VULNERABLE: host02"
}Any host which is patched will be skipped by the “Show Vulnerable Hosts” task. Any vulnerable host will be logged with the message “VULNERABLE: $hostName”.
It would be nicer if there was a way to get the output of vulnerable hosts to not show “ok” but I have not found a good way to change that in Ansible. Basically, the task to “show vulnerable hosts” succeeded, so that is why we see “ok” in the playbook output. As a workaround, I tried setting the task to always fail and ignore errors (so execution of the playbook doesn’t stop), which flags the entire task as red, but the output is more cluttered with debug information so I opted not to use that method.
Hopefully this is helpful to someone!
]]>