Posts for Sunday, October 14, 2018

“Connected” Buch

Als Gegengewicht zu den damals (und auch heute) üblichen Weltuntergangstexten zur Digitalsphäre schrieb ich in den Jahren 2014-2015 alle zwei Wochen eine Kolumne zu unterschiedlichen mehr oder weniger tagesaktuellen Themen.

Die besten und heute noch relevanten Texte aus dieser Zeit sind jetzt als Buch im Ach Je Verlag herausgekommen und sind (*trommelwirbel*) nun käuflich zu erwerben:

<figure class="aligncenter is-resized"><figcaption>Connected Cover</figcaption></figure>

Das Buch kann für 14,99€ direkt beim Verlag erworben werden. Das Ebook schlägt mit 9,99€ zu Buche. Man kann das Buch natürlich auch bei Amazon erwerben (Referallink).

Das Buch richtet sich einerseits an diejenigen, die damals vielleicht einige Texte verpasst haben wie auch an diejenigen, die gerne eine kompakte Sammlung von Texten zu Themen rund um die sozialen Zusammenhänge digitalvermittelter Kommunikation und Lebensrealität zur Verfügung hätten. Oder natürlich auch an alle, die damals – Patreon gabs da noch nicht wirklich – eigentlich die Kolumne gerne unterstützt hätten und das nun gerne nachholen würden.

Posts for Saturday, October 13, 2018

How to create missing favicons for your bookmarks in Firefox Quantum for Linux

Prior to the advent of Firefox Quantum, several add-ons were available that enabled you to replace the favicons of bookmarks in Firefox, or to create custom favicons for bookmarks lacking a favicon. One of my favourite such add-ons was Favicon Picker 3. However, Firefox Quantum currently lacks an add-on that would enable you to insert […]

Posts for Sunday, October 7, 2018

avatar

Medical animations and surgical visualisation with MedFilm

You’re sitting in front of the doctor’s table in a hospital. The doctor has just spent the past half hour explaining the procedure you will undergo to solve a medical problem that you experience. It sounds complicated — there are a few things you have to do to prepare, some foods to watch out for, and a recovery process of a few months afterwards. You will later come home to only be bombarded with a series of questions from your friends and family, who are all curious and have somehow managed to ask the questions which you didn’t think of asking earlier. It also doesn’t help that their native language isn’t English.

Four years ago, this was exactly the problem that Erik Kylen, a small team in Sweden, and myself working remotely set out to solve. The solution was a series of animated videos to explain various medical issues in simple terms. A doctor could use these videos to help guide patients, and patients could then watch these videos from the comfort of their own home. This is MedFilm.

MedFilm Logo

Each video starts with a gentle description of the various body parts involved in the procedure to introduce the required medical terminology. This is followed by an explanation of how these parts relate to the ailment at hand. The patient is then reminded of the various preparatory steps they need to take before the procedure, such as fasting, or drinking fluids. The surgical procedure is then shown, heavily tested to maintain medical accuracy whilst ensuring that the patient does not see anything gruesome. Finally, the video describes the recovery process, and the steps the patient can take to expedite it.

These videos are simple to understand, are accessible with subtitles and translations into many languages, and tailored for specific medical practices in localised hospitals and countries. Each hospital and country has their own preferred ways of doing things, and these videos accomodate that fact.

A doctor from a participating hospital can share a link to their patient, or interactively use the video during the briefing process on a tablet. A patient can later watch it again to refresh their memory, or reshare it with friends and family.

MedFilm surgical videos on various tablets

Let’s see a demonstration video (and yes, videos can be embedded with custom branding into a hospital or clinic’s website!). Below is the video created for an appendectomy. Usually, getting your appendix removed is a pretty safe, standard procedure, and happens pretty soon after you figure out you have a problem. Most people have also heard of it, which makes it a great procedure to demonstrate.

Here’s the video! Click play below and learn about an appendectomy!

<script id="medfilm-player" src="https://medfilm.se/resources/player.js?h=patientinformationsfilmer&amp;id=17&amp;lang=en"></script>

MedFilm is steadily growing and now has a repository of 40 videos covering topics from cardiology to otorhinolaryngology (I’m not a doctor, so to me that’s a very complicated word!), used in clinics across Scandinavia. I’m proud of the service, and happy that it is able to help patients. If you’re interested are are involved in the health industry, you can contact MedFilm here and we can explore opportunities!

The post Medical animations and surgical visualisation with MedFilm appeared first on thinkMoult.

Posts for Monday, October 1, 2018

A brief discussion about package installation times in Gentoo Linux

I thought that perhaps users of binary-based Linux distributions who are contemplating trying out the source-based distribution Gentoo Linux might be interested to know a bit about package installation times in contrast to binary distributions. I am not going to go into great detail here; this is just to give interested people a quick idea […]

Posts for Saturday, September 29, 2018

avatar

TL;DR: Nudge

The 2017 Nobel Prize laureate, Richard H. Thaler, and Cass R. Sunstein present the libertarian paternalism and choice architecture: the authority does not coerce the policy, instead it influences the choosers’ behaviors to make better choices, aka nudge.

Nudge

Why nudge works

Daniel Kahneman, 2011 Nobel Prize laureate, proposed in his book, Thinking Fast and Slow, that we have two modes of thoughts: system 1 and system 2. System 1 is fast, instinctive and emotional, while System 2 is slower, more deliberative, and more logical. From the engineer point of view:

  • System 1 is like deep learning. Million years training data and selective evolution train the system focusing on the responsiveness at the expenses of accuracy.
  • System 2 is more like domain knowledge. The system evolves from the reflection, and it can rationalize its decision based on the logic and insights. It trades the accuracy over responsiveness.

The canonical Eco will analyze the current status, then apply logic rational to evaluate pros and cons, and conclude a decision at the end. Ideally, he or she MAY also review the decision for a retrospect in the future. ALL of these requires significant cognitive power, and our brains are lazy1, we tend to solve more complicated problem with System 1, which incurs many cognitive biases, such as anchoring, herd mentality mentioned in the book.

On the other side, as the choice architecture designer, we can also exploit the biases for the choosers’ good.

How nudge works

In the book, the authors propose several libertarian paternalism to make chooses better off:

Remover bad options If some options result worse outcomes, the choice architecture designer can simply remove them to make a difference.

Make a better default option We can make a better default option, and leverage the status quo bias.

Rethink regulation in the public policy In the United States, a significant portion of health-care cost is spent on the malpractice insurance. The policy maker’s intent is to protect patients’ best interest, but in practice, the ambiguity of “neglect” makes the malpractice lawsuit more or less a lottery, and lawyers take lion shares. We MAY consider to allow the patients to waive the rights to sue for a more affordable health-care practice.


  1. Consider laziness a virtue from the evolution perspective: the body can allocate the energy for alternative use.

Posts for Monday, September 17, 2018

Automatic backup of users’ files on a NAS device to an external USB HDD

One of my Linux machines is a 4-bay server that performs various roles, one of which is as NAS (network-attached storage) for family and visitors’ devices connected to my home network. I had configured each pair of HDDs in a RAID 1 array in order to provide some internal redundancy, but I was nervous about not […]

Posts for Friday, September 14, 2018

Password feedback for sudo

To make sudo show asterisks when you type a password, add this to your Defaults specification:

Defaults pwfeedback

Posts for Tuesday, September 11, 2018

Make syslog-ng at /dev/tty12 colorful

To make syslog-ng at your /dev/tty12 (or whatever tty you use for your logs, and it works to whatever kind of log from syslog-ng afaik), install ccze and use this line in your /etc/syslog-ng/syslog-ng.conf:

destination console_all { program("ccze -r >> /dev/tty12"); };

Posts for Sunday, September 9, 2018

avatar

cvechecker 3.9 released

Thanks to updates from Vignesh Jayaraman, Anton Hillebrand and Rolf Eike Beer, a new release of cvechecker is now made available.

This new release (v3.9) is a bugfix release.

Posts for Wednesday, September 5, 2018

avatar

Improved AppPasswords in Nextcloud 14

The app passwords have been available in Nextcloud for some time now. They were first introduced when we added second factor authentication to Nextcloud, as you still want to have a way to connect your mobile and desktop clients to your account. In the early days this was all manual labor. In the last year we have added support for app passwords to our mobile clients and the desktop client is following soon.

Posts for Tuesday, September 4, 2018

avatar

Destructuring assignments in TypeScript

This is a really quick post to show some examples of destructuring assignments in TypeScript.

First, what is a destructuring assignment in JavaScript?


Basically when you want to assign some data from an array or object into separate variables.

// from an object to a variable
let obj = {foo: 13, bar: 'baz'}; // get the object

let {foo, bar} = obj; // get two variables out of the object

// from an array
let arr = [1, 2, 3, 4]; // get the array
let [a, b, ...others] = arr; // get three variables out of the array

If that's confusing, the MDN link above will explain everything!

The above JavaScript will compile as TypeScript as well, but sometimes when you define a function, there is no type for the parameters yet, so they become 'any'.


// objects
func({a, b}) { ... } // a and b would be 'any'

func({a, b}: {a: number, b: string} { ... } // now a and by have types!

// arrays

func([a, b]: [number, string]) { ... } // a and by have types!

There is more great info here.

Posts for Wednesday, August 29, 2018

Installing Dropbox in Gentoo Linux following the recent restrictions introduced for Dropbox for Linux

In a 2013 post I explained how I installed Dropbox in Gentoo Linux running KDE 4. The Dropbox company has recently imposed some restrictions in the Linux client, so this is to explain what I did to get Dropbox working again in my two Gentoo Linux installations, both using the ext4 filesystem (unencrypted) and, these […]

Posts for Saturday, August 25, 2018

avatar

Job Hunting in 2018

After six years in SurveyMonkey, I decided to explore external opportunities. I felt surreal after staying on the other side of the table for last couple years. Here are my thoughts and lessons learned.

Red Queen’s Race

Six years ago, I read CLRS, and practiced with Cracking the Coding Interview 150, now we have more than 900 problems in leetcode. The backtracking and graph problems which were categorized as “advanced topic” are now quite common in the interview challenges.

Furthermore, the process has also adapted to the new paradigm. For example, the phone screening used to be 30 minutes conversation between the hiring manager and candidate to exchange information regarding the working experience and job responsibility; now it evolves to live coding session with code sharing tool such as codepad. “Talk is cheap. Show me the code.”

The bar is raised.

Problem solving triumphs over algorithm

Fortunately1, most of the companies mainly focus on the problem-solving skills instead of the algorithm or data structures, such as:

  • Data collection, transformation, aggregation, and analysis.
  • Augment a known data structure for a specific use case.
  • Live debug.

Alternatively, they challenge your system design knowledge with open discussion. I think these tests indicate better signals about the engineers’ capability and potential.

Lessons learned

The most valuable lesson I learned from my mistake is never let off your guard. Whether the session is marked as chat or open discussion, its objective is still evaluating your qualification intellectually, mentally and socially.

If the intensive practice is tiresome, why not just amortize it? I mean, it is better to keep your edge sharp all the time, right? Treat the practice as an opportunity of continuous education:

Joining a discussion group / meetup will also help.


  1. Or unfortunate from the return of investment perspective.

Posts for Tuesday, August 7, 2018

Prevent apple OS from creating .DS_Store and other bullshits

I don't think I have to explain about that. Everyone that had to admin an ecosystem that have one or more apple users know how this can be a nightmare in many different scenarios (specially because at some point, MacOS doesn't care even to hide the files anymore when dealing with network shares, external drives and so on, how nice).

Well... this needs to be applied with the MacOS:

Prevent MDS from attempting to Index
sudo touch /Volumes/volume_name/.metadata_never_index

Disable Indexing AND Searching of Volumes (if necessary)
sudo mdutil -i off -d /Volumes/volume_name

Delete existing Spotlight Index (if necessary, to start over, whatever fits you)
sudo rm -rfv /Volumes/volume_name/.Spotlight-V100
sudo rm -rfv /.Spotlight-V100


Disable creating '.DS_Store' bullshit on USB volumes
defaults write com.apple.desktopservices DSDontWriteUSBStores -bool true

Disable creating '.DS_Store' bullshit on network volumes
defaults write com.apple.desktopservices DSDontWriteNetworkStores -bool true

Now that you've cleaned the mess, remove this bullshit from your drives:
find /path/you/want/to/clean/ -name ".DS_Store" -depth -exec rm {} \;

You can also delete the file responsible for custom icons:
find /path/you/want/to/clean/ -name $'Icon\r' -depth -exec rm {} \; 

Posts for Monday, August 6, 2018

How to move a mouse pointer automatically in Linux to simulate user activity

My various Linux installations all have Suspend to RAM enabled with a specified timeout. Sometimes I want to override the timeout; for example if I have left something running in a terminal window or I have left the package manager in a virtual machine upgrading the guest installation. I could of course launch the system’s […]

Posts for Monday, July 30, 2018

avatar

STLM meetup: The Evolution of Database

Lyft Engineer, Lei Chen reviewed the evolution of databases systems in the STLM meetup. Here are the meeting notes with references and my thoughts.

The Principle

The memory hierarchy of the modern computer dictates how fast we can access the data, and how much data can be stored. The database system is designed to leverage the memory hierarchy to optimize for specialized use cases and make tradeoff between the latency, throughput, and capacity.

Single Host Database

Long long ago, the database is designed for single host due to network performance cannot compete with local storage. Using MySQL architecture as an example:

MySQL Architecture

From bottom up, the MySQL composes several tiers:

  • File System provided by the OS.
  • Storage Engine handles the data layout to optimize for read / write or something between.
  • SQL parser, optimizer, executor provides the SQL semantics.
  • Connection pool manages the incoming connections, sockets, file descriptors

There several directions for the scalability:

  • Separate read / write operations.
  • Sharding: push the scalability problem to the business logic.
  • Drop the SQL support, use key-value store instead.
  • Replace the local file system with distributed file system
  • Some or all of above.

Master-Slave replication

The master-slave replication separates the read / write operations:

  • The master and all slaves handle the read operations.
  • Only the master accepts the write operation, it then replicates to the slaves synchronously or asynchronously.5

Strong consistency requires <semantics>R+W>N<annotation encoding="application/x-tex">R + W > N</annotation></semantics>R+W>N, see the eventual consistency article .

Sharding

With sharding, we can partition the database horizontally across multiple isolated instances to improve the I/O performance. The sharding logic is either handled by a middleware or the application.

Consistent hashing is commonly used to cushion the repercussion when partition is added or removed.

Key-value store

It is not-trivial to support SQL semantics, especially transactions1, what if we drop the SQL support for scalability? Without the SQL semantics, the storage engines can be simplified as key-value stores. It is like the hash map, but the data is too big to fit into the memory2.

Hash Indexes

We can maintain a in-memory hash table to map the key to the offset of the data file. Clearly, we cannot update the value in-place, that requires us to re-index all the succeeding keys. The data file is organized as:

  • append only: any put operation will append a new entry, and the index is updated to point to the new entry.
  • segmented: the data file is broken into segments, so we can compact the data file to reclaim used space, and keep the service available.

Bitcask (the default storage engine in Riak) takes this approach.

SSTable, memtable, and LSM-Tree

The hash indexes does not support range selection3, and demand all keys MUST fit into the memory. This shortcomings are addressed by the Sorted String Table, aka SSTable:

  • We maintain a balanced tree in memory4, aka memtable. Once it grows bigger than the threshold, the data is persisted to a segment with sorted keys.
  • The read operation first searches the memtable, then the most recent segments.

Thus, we maintain a sparse index in the memory instead of all keys. This data structure is also known as Log-structured merge-tree or LSM tree.

LevelDB and RocksDB take this approach, and HBase and Cassandra use the similar storage engine.

Distributed File System

With network speed increases, we can replace the local disks with distributed file systems, such Google File System, (see paper). There are many benefits:

  • Better throughput and reasonable latency6.
  • Data integrity and availability are decouple from the DB design.

Case Studies

Aurora

Amazon’s Aurora performs better than the RDS with two magic bullets (See Deep Dive on Amazon Aurora for more details):

  1. The underlying file system is replaced by EBS(hot data) and S3(cold data).
  2. The master only replicate the write-ahead log, aka WAL to avoid massive I/O.

I am kind of confused by the claim that WAL replication is more efficient; as the WAL write amplification is one of main motive that Uber engineer migrated from pgsql to mysql.

BigTable and Spanner

BigTable leverage the Google File System, SSTable, and Paxos consensus protocol (provided by Chubby?)

Spanner is built upon the BigTable, with atomic clock to ensure global consistency.


  1. The transaction MAY eat up half throughput due to locking primitives.

  2. You MAY find more details in Design Data-Intensive Applications, Chapter 3.

  3. This is by design, the hashed key MUST be randomly distributed to avoid collision.

  4. The BigTable uses the skip list instead of balanced tree.

  5. The replicate strategy is determined by the trade-off of the consistency vs. write throughput.

  6. The jupiter network can deliver 1Pbps of bisection bandwidth.

Posts for Sunday, July 22, 2018

Un poème pour Didier

July 22nd, 2018

Une fois, Aimé, avec son capitaine, Didier, a tout gagné.

Ensuite Didier, vingt ans après, son objectif à bien fixé.

Parmi tous les joueurs, une équipe il a dû sélectionné.

Il a pas tardé, que tout le monde lui est venu la contesté.

Patiemment, tous les adversaires il les a éliminé.

Même les sublimes croates, ils ont le mieux essayé.

À la fin, quand même, c'est Didier qui a gagné.

Posts for Thursday, July 19, 2018

Fixing the network device names

     If you are struggled with a distro with systemd for a random reason, you can fix this dumb nonsense of network devices with this command:

ln -s /dev/null /etc/systemd/network/99-default.link

     I don't know what will break doing this, but since I'm testing this crap using a VM, I don't really care (but you should, so beware where are you doing this). I've tested on Debian and Arch.
     Some people reported that most problems with network manager can be fixed when doing this. 

Update: You can also fix this in non-systemd distros using this in your boot: 

 net.ifnames=0 biosdevname=0
avatar

OSCON 2018 Notes

OSCON 2018

I attended the OSCON 2018 in Portland OR on July 18 – 19. Here are my take-away.

Keynote

Tim O’Reilly delivered the keynote, Open source and open standards in the age of cloud AI and asked a profound question: in the age of cloud AI, are the traditional open source allies, — big corporation such as Google, and Amazon, — turning from collaboration to competing against us?

The open source movement emerged from the great minds’ generosity, and endorsed by IBM with other giant corporation. Both parties benefit from the cooperation until the we hit the top of the S-curve. The relationship MAY change from the cooperation to competition. This also happened in the national level, see Why Nation Fails.

Are we there yet? And if so, what shall we do?

The flourishing Kubernetes ecosystem

The Kubernetes has become the de facto container orchestration platform, and the community focuses on other pain points:

Service Mesh

In the microservice era, the monolithic service is decomposed to many smaller, more modular microservices. The complexity of inter-service grows exponentially1 and the availability of your service is on the mercy of all upstream service2. To build a robust distributed system, we have to manage the upstream services with:

  • caching
  • retry
  • timeout
  • rate limiting
  • circuit breaking, etc

We can certainly build them in the client-side libraries, such as Hystrix; but this approach does not scale in the current polyglot environment. Since deploying services becomes so trivial, why not just wrap our service with a proxy with all good bits?

Envoy and Istio take this approach, aka sidecar pattern, and support more advanced features for continuous delivery, such as traffic shaping, and traffic shadowing. See the demo from Red Hat’s Christian Posta for more details.

Canary in the continuous delivery

Darren Bathgate also discussed the canary in the continuous delivery. These are tools used in the four phases of canary:

  1. Blue/Green Deployment: jenkins tags each deployment with BUILD_NUMBER and push the meta data into the kubernetes deployment.
  2. Traffic Shifting: istio then tune the envoy to load balance the blue/ green deployments.
  3. Observation: Prometheus and Grafana can monitor the error rate for the blue/green developments.
  4. Judgement: we can make a decision based on the observation, or automate with tools such as Spinnaker.

Emerging Languages

TypeScript

The TypeScript is highly influenced by the Benjamin C. Pierce’s work, Types and Programming Languages.

  • TypeScript can infer the type from local context for better interoperability with javascript.
  • It introduces new language constructs to handle the polymorphism in the javascript land, such as FooType | BarType and keyof.

Elixir and Phoenix

Jay Hayes build a phoenix knockoff in dazzling 40 minutes, it highly entertaining and sheds some lights of the framework:

  • The handler function is defined as Endpoint.
  • The endpoints are plumbed with Phoenix Router DSL, get, post.
  • The middleware is composed with Phoenix’s Controller. To be honest, I did not quite follow the meta programming under the hood as my mind halted and the stomach called for the lunch.

Case study

Expedia’s journey to the Cloud

The scalability challenge: 750M searches per day, 15B flight searches annually.

Cloud Migration Strategy

  • Invest to drive the rate of change.
  • Know the landscape.
  • Define your cloud native.
  • Form your Guardrail.
  • Getting unstuck: eliminate the ambiguity and apply growth mindset.

Migration Tactics

The services to migrate fall into the following categories:

  • New services
  • Lift and shift: low pain, low gain.
  • Deprecate
  • Replatform: untangle for high rate of change
  • Move and tune: move as quickly as you can, then iterate with new tools.

Understanding the dependency graph is important.

Define your Cloud Native

  1. Embracing cloud service ecosystem, Amazon build services faster than you can adopt.
  2. Internalizing cloud economics, see Cloud optimization circus.
  3. Accommodate security and control from the beginning.
  4. Build resilient service, and test with intentional failure, see Vegas rule.

Resources

These are tools and resources mentioned in the talk worthy a look:

  • codefresh: a continuous delivery platform for Kubernetes, it simplifies the helm management.
  • helm: the package manager for Kubernetes.
  • spiffe: some application security framework beyond my understanding.
  • johnny-five: A javascript framework for IoT programming.
  • Hyperledger: a open source blockchain framework.

  1. Assume <semantics>n<annotation encoding="application/x-tex">n</annotation></semantics>n services without circular dependencies, the root service can call <semantics>n1<annotation encoding="application/x-tex">n - 1</annotation></semantics>n1 upstream services, each in turn can call <semantics>n2<annotation encoding="application/x-tex">n - 2</annotation></semantics>n2 upstream services, and so; thus the complexity is <semantics>n!<annotation encoding="application/x-tex">n!</annotation></semantics>n!, aka exponential growth.

  2. Assume the upstream services availability is 99.99%, your service’s availability falls to 99.9% with total 10 API calls are made.

Posts for Saturday, July 14, 2018

Configuring Lubuntu 18.04 to enable hibernation using a swap file

In an earlier post about Lubuntu 18.04 I stated that hibernation is precluded because the Lubuntu Installer installs the OS with a swap file instead of a swap partition. In fact, even with a swap file it is possible to configure Lubuntu so that hibernation is possible. This is how I did it. 1.  This PC […]

Capitalism Compromises Design

Capitalism is an economic system based upon private ownership of the means of production and their operation for profit.

Software design always has many competing interests that need to be considered, including quality, time and speed, planning, scalability, security, privacy, cost, and profit.

There are many models and methodologies that present preset weightings and preferences for these interests that are generally agreed upon to be acceptable and even good. “Getting Things Done” emphasizes short-term deliverables and speed arguably at the expense of long-term planning, scalability and quality. Agile can be argued to be the same. As models, they aren’t saying don’t plan at all for growth and scaling, and don’t disregard entirely long-term quality, just don’t “overemphasize” them.

Designing and building software is an exercise in making a multitude of design choices at every step and stage, usually informed by overall goals, approaches, and models.

Capitalism, when introduced into the software design and building process, affects almost every facet immediately, and entire solution spaces are thrown away.

Capitalism elevates profit at the expense of every other interest. Arguments such as “We need a way to monetize, we need access to user data, so we can sell it” of which a subset is “we need to monitor user behaviour in our system, with full metrics, so we can know the user and target them with the best ads” become top priorities in design under capitalism and inform all other decisions.

With these arguments and priorities, capitalism throws away entirely most distributed solutions, often calling into question the viability of end to end encryption (especially for data at rest). They compromise privacy dramatically to the point of obliteration and can usually be demonstrated to weaken security.

In an age where we have many examples of the efficiencies of distributed solutions (BitTorrent for data distribution, pre-Microsoft Skype) it’s sometimes a wonder that a decade later from those innovations, P2P and other decentralized designs never come close to even being mentioned in most design discussions.

Design under capitalism favours centralization over decentralization, and it doesn’t like encryption, generally embracing it only as market demand requires it to keep a product viable.

Decentralization isn’t synonymous with either security or privacy or efficiency, but there is a strong overlap and it can more easily be argued to create a more fertile and easy surface to then build privacy into and efficient systems with. If the user stores their own data, they have control over it and can grant access to it when and only when they consent. Decentralization is more about power, from which privacy can more easily flow. It is still possible to build very inefficient decentralized systems, and even easy to build privacy leaking decentralized solutions. However, it is also possible to build decentralized solutions far more efficient than any centralized solution could ever be and decentralization also lays the groundwork that can allow much stronger privacy systems than a centralized solution can ever guarantee.

It really is very simple. Capitalism wants access to user data, which always degrades privacy. To achieve this it often disregards decentralized solutions entirely, which in-turn can affect many other interests negatively such as scalability and efficiency, favouring “simple” and last decade(s) centralized solutions, which then have to scale awkwardly internally to handle the load demanded of them.

This is why I believe in Open and Free Software. Capitalism and monetization pit the design against users and privacy from the start, and as second-order effects can negatively impact other interests, before the design of a system has even started.

This is why when Open Privacy was deciding how to create itself, we all decided for a nonprofit model over a startup, because there was no viable way we could do the privacy-focused, and user consent enforcing work we wanted to inside of any capitalist model. For us, user privacy had to be the top motivating priority, not profit, and inside capitalism, nothing can trump profit.

Posts for Friday, July 13, 2018

The tasting of surströmming

For the uninitiated, Surströmming is an infamous heavily fermented herring.

Below is my experience with it.

Preparations

I “smuggled” (more on this below) it from Sweden a few months ago and on the evening before the Swedish national day1 my brother, a brave (or naïve) soul of a schoolmate of his, and I (not to mention our dog) opened it up near the river. We chose the riverside and the night time strategically, of course.

As was advised to us by a friend, we also took a bucket of water with us. Not – as some may wrongly assume – to vomit into, but to open the tin under water. Due to the fermentation continuing in the tin, it builds up pressure and when you open the tin, it inevitably and violently discharges the bile water. The best way to avoid it spraying your clothes is to open it under water.

The tasting

Since this was an impromptu action, – other than the bucket – we came only half-prepared. As condiments we brought only a little bread, a shallot and three pickled gherkins.

The hint with the bucket was greatly appreciated, as the opening of the tin was the most vile part of the whole experience. So if you plan to try it, do get a bucket! It stopped not only the bile spraying us, but also diluted most of the putrid smell that was caught in the tin.

Once opened and aired, the contents of the tin were actually quite recognisable. Fish fillets swimming in brine. The brine was already brownish and a tiny bit gelatinous, but darkness helped us get past that.

As for the taste and texture, if you ever had pickled herrings before – it is like that on steroids, married with anchovies. Very soft, but still recognisable as fish, extremely salty, and with acidity that is very similar to that of good sauerkraut.

Washing the fish in the pickle jar helped take the edge of – both in sense of smell and saltiness. The onion as well as the pickles were a great idea, bread was a must!

In summary, it is definitely an acquired taste, but I can very much see how this was a staple in the past and how it can still be used in cuisine. As a condiment, I think it could work well even in a modern dish.

We did go grab a beer afterwards to wash it down though.

P.S. Our dog was very enthusiastic about it the whole time and somewhat sullen that he did not get any.

The smuggling

Well, I did not actually smuggle it, per se, but it took me ¾ of an hour to get it cleared at the airport and in the end the actual carrier still did not know about what I was carrying in my checked luggage. The airport, security, two information desks and the main ground stewardess responsible for my flight were all in on it though. And in my defence, the actual carrier does not have a policy against Surströmming on board (most probably because they have not thought about it yet).

As for acquiring this rotten fish in the first place, I saw it in a shop in Malmö and took the least deformed tin (along with other local specialities). When I came to the cash register with grin like a madman in a sweetshop, I asked the friendly young clerk if she has any suggestion how to prepare it, and she replied that she never had it and knows barely anyone of her generation who did, apart from perhaps as a challenge.

hook out → more fish soon ;)


  1. The timing was purely by chance, but fitted perfectly :) 

Posts for Thursday, July 12, 2018

avatar

DFS, BFS and Topological Sort

In this post, we extend the discussion of graph traverse algorithms: breadth-first search, aka bfs; and depth-first search, aka dfs. They try to solve the problem from different angles, more intuitively:

  • bfs circulates the neighborhood until our goal is met, we MAY also find the shortest path with DP, see Dijkstra’s shortest path algorithm.
  • dfs picks one direction in every crossing until we hits the wall, with appropriate state push / pop, we can backtracking ALL possible solution.

Either way, we build the adjacent list first using collections.defaultdict:

from collections import defaultdict

graph = defaultdict(list)
for start, end in directed_edges:
    graph[start].append(end)
    # For non-directed edges:
    # graph[end].append(start)

It is worthy noting that there exist three states for each vertex:

  • not visited: the vertex is unknown in the current state.
  • visiting: the vertex is being visited, but its dependencies are not handled.
  • visited: all done.

dfs is a typical post-order traversal: the node v is marked as visiting at first encounter, and set as visited only if all its successors are visited. Thus, we can use the dfs to detect the cycle.

def dfs(graph):
    visited = defaultdict(int)  # 0: not visited, -1: visiting, 1: visited.
    return all(do_dfs(graph, visited, v) for v in graph)

def do_dfs(graph, visited, v):
    if visited[v] == -1:
        return False  # cycle detected

    if visited[v] == 1:
        return True  # visited already

    visited[v] = -1
    if all(do_dfs(graph, visited, n) for n in graph[v]):
        visited[v] = 1
        return True
    return False

We can apply the same state transition in bfs, aka the three-color encoding in CLRS P594:

def bfs(graph):
    visited = defaultdict(int)  # 0: not visited, -1: visiting, 1: visited.
    for v in graph:
        if visited[v] == 0:
            do_bfs(graph, visited, v)

def do_bfs(graph, visited, v):
    todo = [v]
    visited[v] = -1  # visiting

    while todo:
        vertex = todo.pop(0)
        for c in graph[vertex]:
            if visited[c] == 0:
                visited[c] = -1
                todo.append(c)
        visited[vertex] = 1

The intermediate visiting state does not help the cycle detection, thus we can simplify the state by visiting the vertex’s children immediately after they are enqueued:

def bfs(graph):
    visited = set()
    for v in graph:
        if v not in visited:
            do_bfs(graph, visited, v)

def do_bfs(graph, visited, v):
    todo = [v]
    visited.add(v)  # visit v

    while todo:
        vertex = todo.pop(0)
        for c in graph[vertex]:
            if c not in visited:
                visited.add(c)  # visit child
                todo.append(c)

In general, bfs is a better choice for graph traverse due to that:

  • No recursion, so the size of the problem <semantics>N<annotation encoding="application/x-tex">|N|</annotation></semantics>N is no longer bound by the maximum stack limit.
  • No intermediate visiting state, just visited or not.

Topological Sorting

The topological ordering is defined as reordering the vertices, <semantics>u<annotation encoding="application/x-tex">u</annotation></semantics>u and <semantics>v<annotation encoding="application/x-tex">v</annotation></semantics>v, <semantics>u<annotation encoding="application/x-tex">u</annotation></semantics>u comes before <semantics>v<annotation encoding="application/x-tex">v</annotation></semantics>v for every directed edge <semantics>uv<annotation encoding="application/x-tex">uv</annotation></semantics>uv. More concretely, if vertex <semantics>v<annotation encoding="application/x-tex">v</annotation></semantics>v depends on <semantics>u<annotation encoding="application/x-tex">u</annotation></semantics>u, then <semantics>u<annotation encoding="application/x-tex">u</annotation></semantics>u must be placed before <semantics>v<annotation encoding="application/x-tex">v</annotation></semantics>v. There MAY exist more than one solutions, and obviously, the graph MUST not contain cycles.

Topological sorting can be used to fine the critical path in the scheduling problem, and we can attack the problem with the following algorithms:

Depth-first algorithm

This algorithm leverages the dfs: since all my dependencies MUST be placed after me; it is safe to place non-visited vertex <semantics>u<annotation encoding="application/x-tex">u</annotation></semantics>u to the head after visiting all its children in the dfs fashion.

Kahn’s algorithm

The Kahn’s algorithm takes the bfs approach:

  1. Pick a vertex, <semantics>u<annotation encoding="application/x-tex">u</annotation></semantics>u without reverse dependencies, or incoming edge.
  2. Find all of adjacent vertices, <semantics>vi<annotation encoding="application/x-tex">v_i</annotation></semantics>vi, and remove the edge
  3. If vertex <semantics>vk<annotation encoding="application/x-tex">v_k</annotation></semantics>vk no longer has any reverse dependency, add it to the candidate pool.
  4. Repeat until the candidate pool is empty. Otherwise, fail due to circular dependencies.

Posts for Wednesday, July 11, 2018

Review of some Vahdam’s Masala Chai teas

Masala chai (commonly and somewhat falsely abbreviated to just “chai”) literally means “spice mix tea” – and this is what this review is about. I got myself a selection of Vahdam’s masala chais and kept notes of each one I tried. Some came in the Chai Tea Sampler and others I either already bought before or were a free sample that came with some other order.

Classical CTC BOP

CTC BOP is usually cheaper than more delicately processed whole leaves. Although the common perception is that it is of lower quality than e.g. FTGFOP or even just FOP or OP for that matter, the fact is that they simply a different method with a different outcome. You can get away with breaking cheaper leaves, though, than whole.

Also bare in mind that while BOP is the most common broken leaf grade, there are several more.

It makes for a stronger brew and a more robust flavour– ideal for breakfast teas. The down-side is that it can coat your tongue. But if you want to recycle it, the second steep will be much lighter.

Original Chai Spiced Black Tea Masala Chai

The quintessential masala chai – the strength of the CTC BOP, paired with the classic mix of spices. A great daily driver and a true classic, but for my personal taste a tiny bit too light on the spice.

Ingredients: CTC BOP black tea, cardamom, clove, cinnamon, black pepper

Double Spice Masala Chai Spiced Black Tea

Same as India’s Original Masala Chai above, but with a bigger amount of spice. Of the two I definitely prefer this one.

Ingredients: CTC BOP black tea, cardamom, clove, cinnamon, black pepper

Fennel Spice Masala Chai Spiced Black Tea

Due to the fennel, the overall taste reminds me a lot of Slovenian cinnamon-honey cookies1, which we traditionally bake for Christmas. The odd bit is the cookies do not include the fennel at all, but most of the other spices in a classic masala chai (minus pepper). I suppose the fennel sways it a bit to the sweet honey-like side.

In short, I really liked the fennel variation – could become firm winter favourite of mine.

Ingredients: CTC BOP black tea, fennel, cardamom, clove, cinnamon, black pepper

Saffron Premium Masala Chai Spiced Black Tea

When I saw the package I thought that saffron was more of a marketing gimmick and I would only find a strand or two in the whole 10g package. But no! The saffron’s pungence punches you in the face – in a good way. It felt somewhat weird to put sugar and milk into it, so strong is the aroma.

Personally, I really like it and it does present an interesting savoury twist. It is a taste that some might love and others might hate though.

Ingredients: CTC BOP black tea, cardamom, cinnamon, clove, saffron, almonds

Earl Grey Masala Chai Spiced Black Tea

I am (almost) always game for a nice spin on an Earl Grey. In this case, the standard masala complements the bergamot surprisingly well and in a way where none of the two particularly stand out too much.

The combination works so well that it would feel wrong to call it a spiced-up Earl Grey or a earl-grey’d masala chai. It is a pleasantly lightly spiced, somewhat citrusy and fresh blend that goes well with or without milk.

Ingredients: CTC BOP black tea, bergamot, cardamom, cinnamon, clove, black pepper

Cardamom Chai Masala Chai Spiced Black Tea

Now, this one is interesting because it only has two ingredients – black tea and cardamom. While not as complex in aroma as most others, it is interesting how much freshness and sweetness a quality cardamom pod can carry.

I found it equally enjoyable with milk and sugar or without any of the two.

Ingredients: CTC BOP Assam black tea, cardamom

Sweet Cinnamon Massala Chai Black Tea

Similar to their Cardamom Chai, it is a masala chai with very few ingredients. The cinnamon and cardamom get allong very well and while it lacks the complexity of a full masala/spice mix, it is a very enjoyable blend.

Recommended especially if you like your masala chai not too spicy, but sweet.

Ingredients: CTC BOP Assam black tea, cardamom, cinnamon

Ortodox black

What is described with “orthodox” usually means a whole leaf grade, starting with OP. These are much weaker than CTC, but therefore bring out the more delicate flavours. It is a bigger challenge therefore to make sure spices do not push the flavour of the tea too much into the back-seat.

Because the leaves are whole, as a rule you can get more steeps out of them than of broken leaves.

Assam Spice Masala Chai Spiced Black Tea

The more refined spin on the classic masala chai – with whole leaves of a quality Assam, it brings a smoothness and mellowness that the CTC cannot achieve. Because of that the spices are a bit more pronounced, which in my opinion is not bad at all. The quality of the leaf also results in a much better second steep compared to the CTC.

Most definitely a favourite for me.

Ingredients: FTGFOP1 Assam black tea, cardamom, cinnamon, clove, black pepper

Tulsi Basil Organic Masala Chai Spiced Black Tea

I have not had the pleasure of trying tulsi2 and regarding masala chais, this is a very peculiar blend. The taste of the Assam is quite well hidden behind the huge bunch of herbs. In fact, for some reason it reminds me more of the Slovenian Mountain Tea than of a masala chai.

In the end, the combination is quite pleasant and uplifting.

What I found fascinating is that it tastes very similar both with milk and sugar, and without any of the two.

Ingredients: organic Assam black tea, tulsi basil, cinnamon, ginger, clove, cardamom, black pepper, long pepper, bay leaves, nutmeg

Darjeeling Spice Masala Chai Spiced Black Tea

As expected, the Darjeeling version is much lighter and works well also without milk, or even sugar. Still, a tiny cloud of milk does give it that extra smoothness and mellowness. It is not over-spiced, and the balance is quite well. The taste of cloves (and perhaps pepper) are just slightly more pronounced, but as a change that is quite fun. It goes very well with the muscatel of the Darjeeling.

Ingredients: SFTGFOP1 Darjeeling black tea, cardamom, cinnamon, clove, black pepper

Oolong

Maharani Chai Spiced Oolong Tea

Despite the fancy abbreviation, IMHO the oolong tea itself in this blend is not one you would pay high prices as a stand-alone tea. Still, I found the combination interesting. If nothing else, it is interesting to have a masala chai that can be drank just as well without milk and sugar as with them.

Personally, I found the spice a bit to strong in this blend for the subtle tea it was combined with. I actually found the second steep much more enjoyable.

Ingredients: SFTGFOP1 Oolong tea, cardamom, cinnamon, clove, black pepper

Green

Kashmiri Kahwa Masala Chai Spiced Green Tea

A very enjoyable and refreshing blend, which I enjoyed without milk or sugar. The saffron is not as heavy as in the Saffron Premium Masala Chai, but goes really well with the almonds and the rest of the spices.

When I first heard of Kashmiri Kahwa, I saw a recipe that included rose buds, so in the future I might try adding a few.

Ingredients: FTGFOP1 green tea, cardamom, cinnamon, saffron, almonds

Green Tea Chai

As is to be expected, the green variety of the Darjeeling masala chai is even lighter than its black Darjeeling counterpart. The spice is well-balanced, with cinnamon and cloves perhaps being just a bit more accentuated. This effect is increased when adding milk.

It goes pretty well without milk or sugar and can be steeped multiple times. Adding either or both works fine as well though.

Quite an enjoyable tea, but personally, in this direction, I prefer either the Kashmiri Kahwa or the “normal” Darjeeling Spice masala chais.

Ingredients: FTGFOP1 darjeeling green tea, cardamom, cinnamon, clove, black pepper

hook out → hopefully back to blogging more often


  1. The Slovenian name is “medenjaki” and the closest thing the English cuisine has to offer is probably gingerbread. 

  2. For more about tulsi – or holy basil, as they call it in some places – see its Wikipedia entry

Posts for Sunday, July 8, 2018

Getting the lock screen to work reliably when resuming from suspension in a single-seat, multi-user Lubuntu 18.04 installation

In an earlier post I described my attempt at getting the lock screen to work reliably in the single-seat, multi-user Lubuntu 17.10 installation on my family’s desktop PC. Although the modifications described in that post seemed to improve matters somewhat, users were still not always able to login from the LightDM greeter screen after resuming […]

Planet Larry is not officially affiliated with Gentoo Linux. Original artwork and logos copyright Gentoo Foundation.