Explaining Monoids

Let’s have some fun with F#. Through example, we will discover together what are monoids, and why they could be useful. All the following code can be found online.

Function composition example with integers

We now all agree about what’s function composition. We all know what are integers. So, let’s compose some functions using the set of integers.

As shown in this code, add and minus for the set of integers are functions defining a rule of combining two elements for the set of integers. These functions have two interesting properties: they are associative, and they behave as an identity function when the neutral element is one of the input.

As you can guess, the neutral element for the integer set is 0. You can trust me, or you can just run this code instead:

Let’s wrap up: the set of integers has a rule (add) for combining two elements of the set, this rule is associative and the set has a neutral element (0).

 

Function composition example with hours 

We can also agree on what are hours (positive integers between 1 and 24), so we could compose some functions using the set of hours.

As it happens, we can also defined add and minus operation on hours. It’s a bit more interesting now, because these operations use a modulo, it’s no longer a simple operation. We can still prove associativity and the existence of a neutral element for the set hours on operation add.

We can add to our knowledge that the set of hours has a rule (add) for combining two elements of the set, this rule is associative and the set has a neutral element (24).
 

Monoid land

Monoids are a collection of things (a set) with a rule of combining two elements of this set together (this rule is called the internal law of composition). This composition law has to be associative, and the set has to contain a neutral element (otherwise it is not a monoid, but it could be another mathematical structure like a magma or a semigroup that we won’t detail here, because they have less interest for a programmer).

Does it remember you something? You got it:

  • The integers set with the function add is a monoid, because add is a rule to combine two int together, add is associative, and add is an identity function when it is used with the neutral element of the set which is zero.
  • The hours set with the function add is a monoid, because add is a rule to combine two hours together, add is associative, and add is an identity function when it is used with the neutral element of the set which is 24.

We use simple examples here, but please understand that any law (function) defining the combination of two element of a set, that is associative and behave as an identity function when used with a neutral element, is a valid rule to define a monoid. No matter the name or the complexity of this function. Note also that we call monoid the set with the function. Integers are not monoids. But integers with the function add is a monoid. Hours with the function add is another monoid. You can’t define a monoid without its internal law of composition.

Parallel Monoid

The coolest thing about monoid is the associativity law. And the cool thing with associativity in programming is that it allows parallel streams. Why?
Because if (f°g)°h =f°(g°h), you don’t care which function is executed in which order. How convenient is that for parallel processing?
 

Monoids everywhere

Cyrille Martraire has a great talk to explain why you should look for monoids in your domain. By their very nature, you can assume they have a neutral element, they are composable and parallelizable (because they are associative), and they are based on a set of value objects. You say all that when you say that something is a monoid in your domain.

Monoids are powerful and deserve to be known, as you probably use them already without noticing their existence!

 

What’s composition?

You remember that Functional Programming (FP) is all about function composition? But what’s composition by the way? Let’s explain it here with a few other useful concepts for FP, and go one step further in monads understanding.

Integer composition

A composition of integer in mathematics is a way of writing N as the sum of a sequence strictly positive. Weak composition is for sequence positive or equal to zero. It means that every positive integer admits infinitely many weak compositions. 0 is also known as the neutral element for the set of integers.

Function composition

A function composition is when we apply a function (say f) to the result of another one (say g) to produce a third function (say h). We write it h=g°f. Intuitively, composing two functions is a chaining process in which the output of the inner function becomes the input of the outer function. This is in essence functional programming.

Identity Function

A function that returns exactly its input as the output without any alteration is also known as the identity function. There is a symetry between weak composition of integers and weak composition of functions, because an identity function is a neutral element for the set of functions. Like 0 for integers, it can help in several situations to allow more composition. Which is useful, because we want to compose everything!

Which functions are composable?

Two functions are composable if their Domain and Co-Domain are compatible. Let’s imagine two functions:

  • StringToInt : s (string) -> i (integer) StringToInt is a function mapping a string (Domain) with an integer (Co-Domain).
  • TwoTimes: i (integer) -> o (integer) TwoTimes is a function mapping an integer (Domain) with another integer (Co-Domain).

Can I imagine a composition TwoTimes>>StringToInt? No because TwoTimes output is an integer, and StringToInt input is a string. The Co-Domain of TwoTimes is not compatible with the Domain of StringToInt. But the Co-Domain of StringToInt is compatible with the Domain of TwoTimes. Thus, we could define a correct composition StringToInt>> TwoTimes. Most FP patterns are just a way to wrap data so that Domain and Co-Domain stay compatible, thus composable, despite side effects.

Composable functions properties

If functions are composable, they are associative. It means: g°f(x) = g(x)°f . You know for example that addition is associative.
So if g is +5, f is +10 and if x is 3, we can write: 5+ (3+10) = (5+3) + 10 = 18.
Note that, if two functions have the same Domain and Co-Domain, they are composable, and because they are composable they are also associative. The composition of functions with the same Domain and Co-Domain are called transformations, because mathematicians like to give a name to everything (and for once it seems a logical name).

How powerful is function composition?

In a few words: it allows to express complex things in a simple form. In physics for example, Lorentz Transformations are very famous. Using a series of transformation, Lorentz showed that the speed of light is the same in all inertial reference frames. The invariance of light speed is one of the postulates of Einstein’s Special Relativity theory.

Of course we’re not Einstein, but expressing complex things in a simple form is definitely something interesting for software design.

 

Do you know what’s a function?

I’m so used to functions in my daily job as software programmer that I was sure to know what it is. As often I was wrong, let me tell you why.

Function and relation

I intuitively called a function anything in my code with an input and an output. I suspected that functions without input and/or output where a special kind of functions. But in truth, f: a->b defines a relation between a and b. The Domain is the set containing a, and the Co-Domain is the set containing b.  A relation maps an input from the Domain with an output in the Co-Domain. A relation can be:

Back to code

If a relation can map an element from the Domain with several elements in the Co-Domain, it is not a function. Do I code a lot of relation with different possible outputs for the same input? Yes, any function with side effects when a is an element of the Domain, can produce different output for the same input. They are not functions by the mathematical definition.

It also means that we can’t define a function without defining the Domain and the Co-Domain. If we change the Domain or the Co-Domain, a relation can be a function or not. A famous example is the SquareRoot relation because:

Sqrt : a (|R) -> b (|R) is not a function (indeed, sqrt(4) can be either 2 or – 2).

PositiveSqrt: a (|R+) -> b (|R+) is a function (a subset of squareRoot that return only the positive value)

Note as well that constraining the Domain and the Co-Domain can make the function more specific (injective, bijective, or both).

So what?

Ok my coding function are not mathematical function, why should I care? Let me ask a different question: why do mathematicians care to manipulate functions instead of relations? Because it’s easier to reason about.

In the same way, the more specific the relation is, the more predictable the code will be. If a relation can produce several outputs for the same inputs, the code is not predictable. It means that my relation depends of a more global context, and thus I can’t reason at the function scale without taking this context into account. On the other hand when my relation is a function (usually called pure function in FP, ie without side effects) my code is predictable, and I can think at a function scale without considering the global context.

It is really powerful, because It doesn’t really matter how smart you are if you have the ability to split a complex problem into simple ones.

 

Why Functional Programming?

In this post, I would like to share why I think the functional paradigm is such a big deal, and why it is worthwhile to learn about it, no matter which technology you are using.

Functional Programming Roots

Functional Programming (FP) is deeply rooted in maths, and is intimidating because of that. But I think we underestimate the value of a mathematical approach of software: it allows to express complex problems with a succession of simple statements. When we translate this to IT, the point is to compose complexity to keep software systems predictable, no matter how big they are.

How mathematics works

Mathematics always follow the same pattern. They define something in a general way, like a polygon: “a polygon is a plane figure that is bounded by a finite chain of straight line segments closing in a loop to form a closed polygonal chain”. Then by applying specific laws, they are able to define more specific things: “A triangle is a polygon with three edges and three vertices”. And by adding more laws, we obtain even more specific things, like an equilateral triangle “An equilateral triangle has all sides the same length”. As you will see, FP design patterns can be explained in the same way: we will use something general, and because we will add some laws, it will define a more specific concept. Note that “A monad is just a monoid in the category of endofunctors” follows this pattern as well. It seems complicated because it uses other concepts we’re not familiar with.

Don’t be scared to learn the symbols

Maths are often intimidating due to complex words and abstract symbols. Learning them is a first step to demystify it. Here are some symbols often used to explain FP concepts:

The mathematical notation y=f(x) means for x, the relation f will produce the output y. f: a->b means for an input a, the relation f will produce an output b. We usually use this notation in FP. Of course we need to define the set for a  and b (integers, positive integers, real numbers, string…). The set of a is called the domain of f, and the set of b is called the codomain of f.

g°f means the composition of g and f, in other word the application of f,then g.

f ≡ g means that f is equivalent to g.

Software predictability

In a pure FP program, f: a->b is always true. Functions are pure, which means they will always produce the same output for the same input, no matter when the function is called. In our industry, because states can mutate over time, we build software where it’s false. If f has side effects, or if anyone can mutate a or b, we can no longer predict the output of f, because it is context dependent. It adds a lot of complexity in our code. Building a program by composing pure function can remove this complexity.

Why Function Composition?

Functional programming is all about function composition. If I have 2 function f:a->b and g: b->c, I want the composition g°f : a->c. The composition of simple functions allows to build services. The composition of services can manage use cases. And a composition of uses cases is an application.

Wrap up

Don’t be scared by mathematical symbols, and remember that if we compose pure functions by respecting some mathematical laws, we’ll be able to build a predictable complex software. It is certainly something that has some interest, isn’t it?

 

The post where you don’t learn what’s a monad, but where you may want to learn more about it

Since a while I wander how to write another blog on the internet trying to explain what’s a monad. The problem is well known, as soon as you understand what a monad is, you lose the ability to explain it.
The good news is that I still don’t fully understand what’s a monad, so maybe I’m still able to explain it.
I don’t claim to be a mathematician, or Functional Programming (FP) expert. To be honest I am pretty bad in maths, and I use F# when I can, but I still work with C# most of the time.
Thus if you are an expert and see any errors in the FP related blog posts , please do me a favor and correct me 🙂

 

A combination of Failures

To explain monads, I started a first blog post, to explain the math theory and stay at a theoretical level. My draft was quickly 3 pages long, and unclear, even for me. How deep should I dig into the maths? Ok then, let’s try the practical approach.
I started a new blog post, only to focus on a practical example, to show how we can handle error with something close of the maybe monad. I tried to present my work in a local meetup, and to livecode the explanation. By the way it was a 10-minutes talk. No need to say it was also a failure, I hope the attendees learn some stuff, but they certainly still not understand what’s a monad.

 

What’s the point?

It slowly drives me crazy. I was thinking day and night about which metaphor I could use to explain monads. Something different from the railway metaphor from Scott Wlaschin if possible. I started two more blog posts, and was still unhappy with them.
Weeks were running, and I couldn’t explain monads in the simple way I was looking for.
I tried to figure out at which moment in my learning path I finally understood it. I also challenged the really reason to do that? Why should I care? Why a developer should know what’s a monad?

Why should we care?

Because monads are a basic building block in functional programming. I think this tweet  nailed it.
Semigroup, monoid, applicative and monad are building blocks, or design patterns inspired from mathematics. They allow to combine anything.
And because functional programming is all about function composition,  it is important to know these blocks as a functional programmer. But the concepts of FP are helpful for an Object Oriented (OO) developer in many aspects as well, as uncle Bob recently points out.

 

What’s a monad?

Something hard to understand without the maths roots. That’s why I will realease several articles to explain both technical and mathematical aspects, this way you can choose which part is relevant for you. From basic maths to concrete example, choose your path to monads understanding!

 

In defence of defence of Software Craftsmanship and Uncle Bob

I hope some water has gone under the bridge after the “Google memo gate” followed by the “Uncle Bob is a misogynist gate”, so let’s discuss it again. During a few days, I feel almost uncomfortable to read Twitter. I read many anger reactions, many accusations and a few smart comments. I’d like to share my feelings in this post, because I believe I’m not the only one to feel bad about this thread.

In defence of software craftsmanship

I’d like to thank Daniel Irvine to have written a good post to defend software craftsmanship. I won’t rephrase here what he already said, but I would like to testify as a software craftsman. I came to this community explicitly because I was not looking only for technical excellence. And I think it is the case for most crafts(wo)men I know. It is an open minded and friendly community.
I feel that even more due to recent events, because at every new attacks in Paris, London, Barcelona or Turku, I can think to crafts(wo)men living here, and I hope nothing bad has happened to them. If I care about them it’s because I know we share some values and principles. Not only the quest for technical excellence, but also a deep respect for people, a willing to improve the things around them and to share their passion. I feel connected to them.

The software craftsmanship community is by far the place where I’ve met the more activists. They talk a lot about ethics, politics, environment, and of course diversity. I can’t remember a single SoCraTes I attended where several of these topics were not discuss. And it is important to me, because I know I must improve about them. Of course, it is not perfect, it is often the same people who talk about these topics. But please don’t assume that other people don’t talk about that because they are not interested. They don’t talk about that because like me, they are often part of a favourited majority, and don’t feel legitimate to discuss such topics. I know it’s wrong, I don’t try to justify it, I just try to explain how I feel, and why I need to change.
As a software craftsman, I would like to testify that this community is certainly not a bunch of old white men who care about TDD. To be honest, this is the community where I see the most diversity, in terms of country, sex, religion, and skin colours. I agree it is still not enough. But it is unfair to say that this community only care about technical stuff.

In defence of Uncle Bob

But I think we missed a post to defend Uncle Bob as well. I’d like to thank him for the many good, talks, books and blog articles he has written. He was the one who enlights the passion in me with Clean Code.
Of course, he has done some bad stuff as well (but who doesn’t?), and I don’t agree with every word he says. For instance, I don’t believe working 70 hours per week is mandatory to be a good software engineer. And no need to say I don’t believe that you should not fire someone who has toxic ideas.
But his point was more subtle: “Don’t fire someone just because you disagree with him”. I think his example and context was bad, but it raises a fundamental question: how do we differentiate toxic ideas and ideas we disagree with?  It’s a fair question because if we don’t agree with toxic ideas, aren’t we tempted to label as “toxic” any idea we might not like? How do we react with people who label agility, scrum, XP or craftsmanship as toxic?

I don’t have the answers, but my point is that the community has overreacted. Yes, the example was poor, yes, the context was bad. But instead of asking for explanations (that he writes here), and discussing the core question (how do we differentiate toxic ideas and ideas we disagree with?), there was lots of anger and unrational discussions about since when and how much Bob is a misogynist, and if we should refuse him to speak in conferences. Aren’t we open-minded? Aren’t we here to help each other? Uncle Bob is a master of technical stuff, but he says that he needs to improve on the topic of diversity. Like me, and like many other average white men in the industry.
The solution is not to ignore him and label his speech as “toxic”. The solution is to discuss, argue and teach, because we are all smart software engineers who share values and principles, including the willing to improve, and not only technically.

As a software craftsman, I would like to testify that this community is very open minded, and that it can help people to improve, even Uncle Bob.

 

6 things to enjoy coding

I really enjoy my job. I understand some people code for food, and it’s fine. But some passionate people are also frustrated. For them I’d like to share some tips. I think most of them can be applied whatever your job is, but I can only testify as a developer.

Ease of mind
I need to be focus on the problem to solve to
be productive. Thus I use several techniques to free my mind at work:
– I don’t hesitate to interrupt myself to fix an important personal problem
– I try to do 1h of workout per day, just before lunch
– I strive to avoid misunderstanding with my co-workers and customers using a lot of communication (never keep a hard feeling for myself)
I may practice some meditation on very bad days

Small cognitive load
When a problem is too hard, I fix an easier part of the problem. Then I iterate until the whole problem is solved. Practices like TDD and EventStorming are a concrete implementation of this concept.
No need to be a genius to fix pretty complex problem thanks to this method. It requires “only” discipline and practices.

Focus sprint
I avoid multi tasking as much as possible. I saw several times that multi tasking is the best friend of workaholics. It is tempting to show how busy we are, but most of the time, it is just bad time management.
My strategy is to work less, but with high focus.
I use pomodoros to time-box 25 minutes sprints. During a sprint, I have a headphone on my ears and all notifications are shut down. On a typical day, I achieve 6 to 8 pomodoros, the rest of the time
I manage interruption.

Sharing

I do one pomodoro each week to share on my blog and on StakOverflow. I’m also really involved in local communities in Lyon. I attend 4 to 8 conferences per year (mainly in France but also in Europe).
I learn much more since I took this habit to share. Sometimes I’m wrong. Sometimes I sound stupid. And I know my English is far from perfect. But it’s fine as long as I learn something on the way.

Getting out of our comfort zone

If your the smartest person in the room, you’re in the wrong room”
L
ook for something to learn in the place you are, or go somewhere else. There is always something new to learn in our amazing job: a domain, a language, an IDE, a method, a paradigm…
Whatever it is, it will help you to be a better programmer.
I’m use to track my goals, and to evaluate them twice a year to see if I’m still moving forward to my path to become a software craftsman.

Have fun

Thanks to remote working 4 days per week, I can spend lots of time with my family, without give up workouts, video games, side projects, conferences and user groups.
Working is not your life, it’s a mean to enjoy it. Thus take time for your family, friends, and to have fun. There is no way to enjoy your job if you can’t enjoy your life. 


 

What the fuck ORM means?

Today is another day where a data model lealking into my business model ruins my productivity. So let’s talk again about impedance mismatch please.

Relational data model

We are all used to relational database. Back in school we learned how to design 1-n or n-n relationships and how to manage primary and foreign keys to manage consistency. We also learned how to normalize and denormalize the model, and to use views for performances.
What we did not learned in school is that it is only one way to store data, and that an automatic translation from this data storage model into memory is not a great idea.

What the fuck ORM means?

According to wikipedia: an Object Relational Mapper is a programming technique for converting data between incompatible type systems in object-oriented programming (OOP) languages.
The problem lies in the definition: we have incompatible type systems. But we somehow believe that the storage data model and the in memory object oriented model are the same thing.

The mismatch

Relational and object oriented are two different representation of the same model. One uses keys and relations to achieve data integrity, based on relational algebra and tuple relational calculus . The other defines bounded objects with defined responsibilities to simulate business processes in order to take decision, inspired by how cells work in biology.
Loading a relational model into memory as an object grape leads to well known problems. Lack of encapsulation (public get/set) is one of them. Anaemic model (data object without any behaviours) is another one. Both of them violate OOP principles.
There are also technical problems: when do you stop to load the grape? How do you manage circular references?

Complexity increase

In all the projects I met so far, the benefice of using an ORM was balanced by the incredible number of hacks we used to load and save what we need. Not speaking about the unmanageable complexity as soon as we try to cache something, because it is always a nightmare to know what is in memory or not.
Just to be clear: the core problem is not SQL or ORM by themselves (they are just tools), but the fact that most developers consider the data model to be their domain model.

How to escape it?

Keep in mind that a relational model is for storage, and that there is no simple way to automatically convert this model in your object oriented representation.
Using other data storage (document, graph, events…) mights help to remind that storing data and running business processes are two different problems with two different purposes.
And if a SQL storage is choose, I encourage to use simple library like Dapper to write actual SQL requests in the data layers. It will simplify the code, clarify the layers responsibilities and learn some SQL to the team, which is a great idea.

Trying to match automatically data model and domain model is a failure, because they are two different concepts.

 

It’s all about fast feedback

I’m a big fan of James Coplien.
I appreciate his controversial point of view about Unit Testing, even if I don’t agree with him. Recently he tweeted about this article explaining why integration tests are much better than unit tests.
Let’s explore it together.

Unit test problem

You write code and then run the unit tests, only to have them fail. Upon closer inspection, you realize that you added a collaborator to the production code but forgot to configure a mock object for it in the unit tests.”

In other words he points out that after a refactoring, some tests will fail despite there is no behaviour modification.
But it really depends on how you write your unit tests. When practicing TDD, I’m testing a logical behaviour. Several classes can emerge, I don’t necessarily have one test class for one production class. And when coupled with DDD practices, I can write unit tests without mocking anything, because I test business logic. Also when I strive for applying SRP, I have a few functions per interfaces, meaning one test won’t fail just because I change some interfaces in another place.
That being said, I agree that some refactoring might change some contracts, which requires to change both the production and the test code. My point is: if it is complicated, it is not because of the unit tests.

System tests solution

“While I was enslaved by unit tests, I couldn’t help but notice that the system tests treated me with more respect. They didn’t care about implementation details. As behavior remained intact. they didn’t complain.”
(System tests for the author are almost end to end and include database)

Again, I agree, if it’s a technical refactoring, the tests that use only production code without mocking may be less brittle.
But I also notice that, when a system test goes red after an actual regression, it takes me a while to find out what the actual problem is. Also, when I had some bugs in production, it is harder to reproduce it in my system tests. It encourages me to fix the code without changing the system tests.
And finally, when there is a business change, it hurts much more to change my system tests. I need a few hours just to write the test managing the new behaviour. I have to initialize half of the application just to check 2 lines of code.

 

Pyramid tests is a myth!

“If you’re like me, you were trained to believe in and embrace the testing pyramid.”…”But my experience revealed a different picture altogether. I found that not only were unit tests extremely brittle due to their coupling to volatile implementation details, but they also formed the wider base of the regression pyramid. In other words, they were a pain to maintain, and developers were encouraged to write as many of them as possible.”

I definitely disagree.
I was not trained to believe in the testing pyramid. I was trained to believe in the “get your shit done, faster please”. And using this approach I believed, like the author and many other developers, that system tests have a better ROI.

It’s all about Fast Feedback

So I tried, and it doesn’t work for me because of long feedback (more than a few minutes).
Feedback time is everything when you run your tests hundreds of times per day, on your local machine. And running your tests hundreds of times per day is really important to practice TDD.
Running the tests has to be easy and fast, or it becomes a pain. And when it’s painful, we loose discipline. We start checking Twitter while the tests are running, and that’s the start of the end of our productivity.


Contextual testing

Maybe there are contexts where systems tests have a better ROI than unit tests. But in my context of backend complex application, with lots of business requirements changing every day, it is not the case.
And I’m not speaking theoretically here. I was involved in 6 projects since I started my career. One has no tests at all, two used inverse pyramid, and three used the classic Mike Kohn Pyramid.
No need to talk about the failure of the project without tests.
The two who used the inverse pyramid have commons characteristics: it was really painful to maintain after a few months, and we need one or two developers almost full-time to keep the system tests working. Not mentioning the time we loose to run the tests before pushing our code, meaning that we often pushed without testing and break the build.

The projects I enjoyed the most as a developer, and my customer as well I believe, was when we have a solid unit testing strategy following the classical Mike Kohn Pyramid. We were able to run thousands of unit tests in a few seconds, and keep a few integration tests running only on the server.

To get results in the long run, don’t give up unit tests, at least if you are working in the same context than me.

 

The coding Iceberg

Recently, I read a few posts about AIs, including Uncle Bob’s post, in response to Grady Booch (about this tweet).
I think the fact tha
t some people believe AIs might replace developers leads to interesting questions about what is programming?

What is programming?

As Bob recalls us, Alan Turing defined programming as a sequence of simple statements of the following form:

Given that the sstem is in state s1,
When event E occurs,
Then set the system to state s2 and invoke behavior B

We express some needs in terms of state transition to be executed by a machine, in other words we build state machines in order to satisfy precise specifications.
This explanation makes it crystal clear that the complexity increases with the number of states in the system.

The perception

Lots of people consider coding as a translation in a given language. You can translate a text from French to English, so you should be able to translate specifications from human readable to computer understandable.
Of course, it is only the emerged part. The translation from human readable to machine understandable code is already automatic, it’s called the compilation.

Developers must be Domain Driven

Under water, developers must express business needs in an unambiguous way, which is the hard part. Coding would be a translation only if the requirements were already a precise state machine, ie a suite of exhaustive Given/When/Then.

As the specifications are never precise enough, we developers must complete them. We have to challenge existing specifications with our understanding of computer, and we have to fill the missing parts with our understanding of the domain.
Thus, considering developers as simple translator who do not need to understand what they are writing about is a sure recipe for failure.

The future

Believing that coders can be replaced by AIs is believing that coders do not need to understand what they are writing about.

AIs might help developers to do a better job though, if they are designed to optimize automatic part like compilation.

 
IP Blocking Protection is enabled by IP Address Blocker from LionScripts.com.