Luis Orduzhttps://luord.com/2023-12-02T02:09:16-05:00Software Engineerhttps://luord.com/assets/img/site/favicon.pngA genetic algorithm implemented in Python2023-03-01T00:00:00-05:002023-12-02T02:09:16-05:00Luis Orduztag:luord.com,2023-03-01:/2023/03/01/genetic<p>An example of abstraction and analogy.</p><p>Natural selection is, roughly, the likelihood of a given individual to survive long
enough to reproduce, and thus continue its species. Factor in mutations—random changes in the
genes—and the probability a given mutation has to help an individual survive (or not) in its
environment and the result is that some individuals are more likely to reproduce than others.
Those fitter individuals are more likely to pass on their mutations to the next generation, which will add
mutations of its own, ultimately causing the population to slowly change as these mutations
accumulate. Repeat this process over multiple generations across millions of years and we
get evolution.</p>
<p>Turns out that implementing these ideas, or at least analogies, in software can be useful to
solve certain problems, so let's write a simple program that exemplifies the process.</p>
<h2>Seed</h2>
<p>There are multiple types of genetic algorithms with multiple different uses, but usually they
start with a data sample.</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">collections.abc</span> <span class="kn">import</span> <span class="n">Collection</span>
<span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Protocol</span>
<span class="k">class</span> <span class="nc">Individual</span><span class="p">(</span><span class="n">Collection</span><span class="p">,</span> <span class="n">Protocol</span><span class="p">):</span>
<span class="o">...</span>
<span class="k">class</span> <span class="nc">Population</span><span class="p">(</span><span class="n">Protocol</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">select_random</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">Individual</span><span class="p">:</span> <span class="o">...</span>
<span class="k">def</span> <span class="nf">algorithm</span><span class="p">(</span><span class="n">population</span><span class="p">:</span> <span class="n">Population</span><span class="p">):</span>
<span class="n">parent_a</span> <span class="o">=</span> <span class="n">population</span><span class="o">.</span><span class="n">select_random</span><span class="p">()</span>
<span class="n">parent_b</span> <span class="o">=</span> <span class="n">population</span><span class="o">.</span><span class="n">select_random</span><span class="p">()</span>
</code></pre></div>
<p>To keep things simple, we start with two parents that are selected randomly from the existing
population, and we'll go from there.<sup id="fnref:protocols"><a class="footnote-ref" href="#fn:protocols">1</a></sup></p>
<h2>Crossover</h2>
<p>With our first pair in place, we can now produce the next "generation".</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">TypeVar</span>
<span class="c1"># ...</span>
<span class="n">Ind</span> <span class="o">=</span> <span class="n">TypeVar</span><span class="p">(</span><span class="s1">'Ind'</span><span class="p">,</span> <span class="n">contravariant</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">bound</span><span class="o">=</span><span class="n">Individual</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">Offspring</span><span class="p">(</span><span class="n">Protocol</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">mutate</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">Individual</span><span class="p">:</span> <span class="o">...</span>
<span class="k">class</span> <span class="nc">Population</span><span class="p">(</span><span class="n">Collection</span><span class="p">,</span> <span class="n">Protocol</span><span class="p">[</span><span class="n">Ind</span><span class="p">]):</span>
<span class="c1"># ...</span>
<span class="k">def</span> <span class="nf">crossover</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">first</span><span class="p">:</span> <span class="n">Ind</span><span class="p">,</span> <span class="n">second</span><span class="p">:</span> <span class="n">Ind</span><span class="p">)</span> <span class="o">-></span> <span class="n">Offspring</span><span class="p">:</span> <span class="o">...</span>
<span class="k">def</span> <span class="nf">add</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">new</span><span class="p">:</span> <span class="n">Ind</span><span class="p">):</span> <span class="o">...</span>
<span class="k">def</span> <span class="nf">algorithm</span><span class="p">(</span><span class="n">population</span><span class="p">:</span> <span class="n">Population</span><span class="p">):</span>
<span class="c1"># ...</span>
<span class="n">base_offspring</span> <span class="o">=</span> <span class="n">population</span><span class="o">.</span><span class="n">crossover</span><span class="p">(</span><span class="n">parent_a</span><span class="p">,</span> <span class="n">parent_b</span><span class="p">)</span>
<span class="n">real_offspring</span> <span class="o">=</span> <span class="n">base_offspring</span><span class="o">.</span><span class="n">mutate</span><span class="p">()</span>
<span class="n">population</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">real_offspring</span><span class="p">)</span>
</code></pre></div>
<p>The <strong><em>crossover</em></strong> in genetic algorithms is the operation used to combine the data of
the parents to produce offspring. But we can't just stop there, we need genetic variance to ensure
the population actually evolves over time. One form of variance is of course that the parents
are shuffled and a random number of genes is picked from each parent, but even that isn't enough as
it could leave us stuck<sup id="fnref:pool"><a class="footnote-ref" href="#fn:pool">2</a></sup>.</p>
<p>Actual variance comes from the key element of <strong>mutation</strong>, the random chance that any given
offspring individual will have genes not present in the parents.<sup id="fnref:offspring"><a class="footnote-ref" href="#fn:offspring">3</a></sup></p>
<p>Finally, the new individual is, of course, a new member of the population so we add it<sup id="fnref:type"><a class="footnote-ref" href="#fn:type">4</a></sup>.</p>
<h2>Natural Selection</h2>
<p>At this point, we have parents and their offspring, what now? It's time to determine the goal.
Genetic algorithms are commonly used to find a good enough solution to certain types of,
often trial and error, problems that don't translate well to common normal algorithms.
Fortunately, the only thing resembling a "goal" in nature is simply thriving, surviving long
enough to reproduce...
So let's do that, by introducing a "niche" and determining how well the individuals fit that niche.</p>
<div class="highlight"><pre><span></span><code><span class="c1"># ...</span>
<span class="k">class</span> <span class="nc">Population</span><span class="p">(</span><span class="n">Collection</span><span class="p">,</span> <span class="n">Protocol</span><span class="p">[</span><span class="n">Ind</span><span class="p">]):</span>
<span class="c1"># ...</span>
<span class="k">def</span> <span class="nf">remove</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">individual</span><span class="p">:</span> <span class="n">Ind</span><span class="p">):</span> <span class="o">...</span>
<span class="k">class</span> <span class="nc">Niche</span><span class="p">(</span><span class="n">Protocol</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">tournament</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">pop</span><span class="p">:</span> <span class="n">Population</span><span class="p">)</span> <span class="o">-></span> <span class="nb">tuple</span><span class="p">[</span><span class="n">Individual</span><span class="p">,</span> <span class="n">Individual</span><span class="p">]:</span>
<span class="o">...</span>
<span class="k">def</span> <span class="nf">algorithm</span><span class="p">(</span><span class="n">population</span><span class="p">:</span> <span class="n">Population</span><span class="p">,</span> <span class="n">niche</span><span class="p">:</span> <span class="n">Niche</span><span class="p">):</span>
<span class="c1"># ...</span>
<span class="n">fittest</span><span class="p">,</span> <span class="n">unfit</span> <span class="o">=</span> <span class="n">niche</span><span class="o">.</span><span class="n">tournament</span><span class="p">(</span><span class="n">population</span><span class="p">)</span>
<span class="n">population</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">unfit</span><span class="p">)</span>
</code></pre></div>
<p>Nature is ruthless, and so is our algorithm. In nature, only the fittest perpetuate their
genes, and in our algorithm, only that individual that best fits the niche is the
one to continue. This is usually called "<strong><em>tournament selection</em></strong>" in genetic algorithm jargon.</p>
<p>Finally, to maintain our analogy (and really to prevent our population from growing without bound)
we remove the least fit individual from the population.</p>
<h2>Generations</h2>
<p>We have almost completed the algorithm, but the mere fact that we've found an individual that fits
the niche better than others doesn't mean we've actually found one that <em>thrives</em> in the niche; the
likelihood of achieving that in just the first generation is nil. We'll need many generations, so we need
to repeat the process until we find such individual.</p>
<div class="highlight"><pre><span></span><code><span class="c1"># ...</span>
<span class="k">class</span> <span class="nc">Population</span><span class="p">(</span><span class="n">Collection</span><span class="p">,</span> <span class="n">Protocol</span><span class="p">[</span><span class="n">Ind</span><span class="p">]):</span>
<span class="c1"># ...</span>
<span class="k">def</span> <span class="nf">find_mate</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">individual</span><span class="p">:</span> <span class="n">Ind</span><span class="p">)</span> <span class="o">-></span> <span class="n">Individual</span><span class="p">:</span> <span class="o">...</span>
<span class="k">class</span> <span class="nc">Niche</span><span class="p">(</span><span class="n">Protocol</span><span class="p">[</span><span class="n">Ind</span><span class="p">]):</span>
<span class="c1"># ...</span>
<span class="k">def</span> <span class="nf">can_thrive</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">individual</span><span class="p">:</span> <span class="n">Ind</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span> <span class="o">...</span>
<span class="k">def</span> <span class="nf">algorithm</span><span class="p">(</span><span class="n">population</span><span class="p">:</span> <span class="n">Population</span><span class="p">,</span> <span class="n">niche</span><span class="p">:</span> <span class="n">Niche</span><span class="p">)</span> <span class="o">-></span> <span class="nb">int</span><span class="p">:</span>
<span class="n">parent_a</span> <span class="o">=</span> <span class="n">population</span><span class="o">.</span><span class="n">select_random</span><span class="p">()</span>
<span class="n">parent_b</span> <span class="o">=</span> <span class="n">population</span><span class="o">.</span><span class="n">select_random</span><span class="p">()</span>
<span class="n">generations</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">while</span> <span class="ow">not</span> <span class="n">niche</span><span class="o">.</span><span class="n">can_thrive</span><span class="p">(</span><span class="n">parent_a</span><span class="p">):</span>
<span class="n">base_offspring</span> <span class="o">=</span> <span class="n">population</span><span class="o">.</span><span class="n">crossover</span><span class="p">(</span><span class="n">parent_a</span><span class="p">,</span> <span class="n">parent_b</span><span class="p">)</span>
<span class="n">real_offspring</span> <span class="o">=</span> <span class="n">base_offspring</span><span class="o">.</span><span class="n">mutate</span><span class="p">()</span>
<span class="n">population</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">real_offspring</span><span class="p">)</span>
<span class="n">fittest</span><span class="p">,</span> <span class="n">unfit</span> <span class="o">=</span> <span class="n">niche</span><span class="o">.</span><span class="n">tournament</span><span class="p">(</span><span class="n">population</span><span class="p">)</span>
<span class="n">population</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">unfit</span><span class="p">)</span>
<span class="n">parent_a</span><span class="p">,</span> <span class="n">parent_b</span> <span class="o">=</span> <span class="n">fittest</span><span class="p">,</span> <span class="n">population</span><span class="o">.</span><span class="n">find_mate</span><span class="p">(</span><span class="n">fittest</span><span class="p">)</span>
<span class="n">generations</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">return</span> <span class="n">generations</span>
</code></pre></div>
<p>There are several options here, one commonly used in real genetic algorithms is to pick the two
fittest instead of just one and make those "reproduce", producing an entirely new population
and keep iterating from there. But to keep our
natural analogy going, let's instead assume that our fittest finds instead a "suitable mate"<sup id="fnref:mate"><a class="footnote-ref" href="#fn:mate">5</a></sup> in another
member of the population, which also adds another source of variance.</p>
<p>Ultimately, the point here is iteration: continually doing the crossover and tournament selection until we meet
our goal.</p>
<hr>
<p>And there we have it, that <code>algorithm</code> function represents our full genetic algorithm, in a way I hope is self-explanatory
enough. That function <em>should</em> work without change as long as it receives arguments that actually implement
the <a href="https://peps.python.org/pep-0544/">protocols</a> properly.</p>
<p><a href="https://luord.com/assets/code/genetic/definition.py">Here's a file</a> with the complete definition, and <a href="https://luord.com/assets/code/genetic/implementation.py">here's a file</a> with a
string-based implementation of the algorithm, along with the function being run.</p>
<p>Now, before finishing, you'll notice that I talked very little about the actual <em>problems</em> that could
be solved with this type of algorithm... Well that's true, because the point of this post was the
algorithm itself. That said, I might write a follow up with a practical example.</p>
<h2>Appendix (On implementation and testing)</h2>
<p>I mentioned above that implementation doesn't matter and it indeed doesn't but for the sake
of completeness—to fully explain the genetic algorithm—I wanted to go over what happens during
crossover and tournament selection. However, we can write <a href="https://luord.com/assets/code/genetic/test.py">tests</a> to do that instead
of explaining the implementations line by line!<sup id="fnref:jokes"><a class="footnote-ref" href="#fn:jokes">6</a></sup></p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">hypothesis</span> <span class="kn">import</span> <span class="n">given</span><span class="p">,</span> <span class="n">strategies</span> <span class="k">as</span> <span class="n">st</span>
<span class="kn">from</span> <span class="nn">implementation</span> <span class="kn">import</span> <span class="n">Individual</span>
<span class="n">st</span><span class="o">.</span><span class="n">register_type_strategy</span><span class="p">(</span><span class="n">Individual</span><span class="p">,</span> <span class="n">st</span><span class="o">.</span><span class="n">builds</span><span class="p">(</span><span class="n">Individual</span><span class="p">,</span> <span class="n">st</span><span class="o">.</span><span class="n">text</span><span class="p">(</span>
<span class="n">alphabet</span><span class="o">=</span><span class="n">Individual</span><span class="o">.</span><span class="n">POOL</span><span class="p">,</span>
<span class="n">min_size</span><span class="o">=</span><span class="n">Individual</span><span class="o">.</span><span class="n">LENGTH</span><span class="p">,</span>
<span class="n">max_size</span><span class="o">=</span><span class="n">Individual</span><span class="o">.</span><span class="n">LENGTH</span>
<span class="p">)))</span>
</code></pre></div>
<p>Before anything is done, we have to tell <a href="https://hypothesis.readthedocs.io/en/latest/">hypothesis</a> how to create an <code>Individual</code> that
actually fits our implementation. Only then we move onto the tests:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># ...</span>
<span class="kn">from</span> <span class="nn">implementation</span> <span class="kn">import</span> <span class="n">Population</span>
<span class="nd">@given</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">test_crossover</span><span class="p">(</span><span class="n">parent_a</span><span class="p">:</span> <span class="n">Individual</span><span class="p">,</span> <span class="n">parent_b</span><span class="p">:</span> <span class="n">Individual</span><span class="p">):</span>
<span class="n">offspring</span> <span class="o">=</span> <span class="n">Population</span><span class="p">()</span><span class="o">.</span><span class="n">crossover</span><span class="p">(</span><span class="n">parent_a</span><span class="p">,</span> <span class="n">parent_b</span><span class="p">)</span>
<span class="n">is_parent_a</span> <span class="o">=</span> <span class="n">is_parent_b</span> <span class="o">=</span> <span class="kc">False</span>
<span class="k">for</span> <span class="n">gene</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">offspring</span><span class="p">,</span> <span class="n">parent_a</span><span class="p">,</span> <span class="n">parent_b</span><span class="p">):</span>
<span class="k">assert</span> <span class="n">gene</span> <span class="ow">in</span> <span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">)</span>
<span class="n">is_parent_a</span> <span class="o">|=</span> <span class="n">gene</span> <span class="o">==</span> <span class="n">a</span>
<span class="n">is_parent_b</span> <span class="o">|=</span> <span class="n">gene</span> <span class="o">==</span> <span class="n">b</span>
<span class="k">assert</span> <span class="n">is_parent_a</span> <span class="ow">and</span> <span class="n">is_parent_b</span>
</code></pre></div>
<p>This test tells us everything we need to know about what
happens in <code>crossover</code> without actually having to check the implementation: We don't care how it's
done, but we do care that every gene in the offspring comes from one of the parents, and that <em>both</em>
parents had an input.</p>
<div class="highlight"><pre><span></span><code><span class="c1"># ...</span>
<span class="kn">from</span> <span class="nn">implementation</span> <span class="kn">import</span> <span class="n">Niche</span>
<span class="nd">@given</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">test_tournament_selection</span><span class="p">(</span><span class="n">niche</span><span class="p">:</span> <span class="n">Niche</span><span class="p">,</span> <span class="n">population</span><span class="p">:</span> <span class="n">Population</span><span class="p">):</span>
<span class="n">pop</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">population</span><span class="p">)</span>
<span class="n">winner</span><span class="p">,</span> <span class="n">loser</span> <span class="o">=</span> <span class="n">niche</span><span class="o">.</span><span class="n">tournament</span><span class="p">(</span><span class="n">pop</span><span class="p">)</span>
<span class="k">while</span> <span class="nb">len</span><span class="p">(</span><span class="n">pop</span><span class="p">)</span> <span class="o">></span> <span class="mi">1</span><span class="p">:</span>
<span class="k">assert</span> <span class="n">winner</span> <span class="ow">in</span> <span class="n">pop</span>
<span class="n">pop</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">loser</span><span class="p">)</span>
<span class="n">_</span><span class="p">,</span> <span class="n">loser</span> <span class="o">=</span> <span class="n">niche</span><span class="o">.</span><span class="n">tournament</span><span class="p">(</span><span class="n">pop</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">assert</span> <span class="n">winner</span> <span class="o">==</span> <span class="n">loser</span>
</code></pre></div>
<p>Tournament selection is a bit trickier to test because the calculation for each fittest <em>is</em> an
implementation detail, but one the whole idea depends upon. We could simply repeat the implementation
here and assert that the winner and loser were calculated correctly but then the test would no
longer work if we changed the metric (or changed what an individual is entirely).</p>
<p>In these cases, we have to step back and think of invariants: what is always true about the tournament
selection? As long as the winner remains in the population and no other individuals are added, it will <em>always</em> be the winner. And that's what we test: We systematically remove each loser until only one
individual is left in the population; that individual <em>must</em> still be the original winner.</p>
<div class="footnote">
<hr>
<ol>
<li id="fn:protocols">
<p>I use protocols because Python's <a href="https://peps.python.org/pep-0544/">structural subtyping</a> is pretty good at
properly representing a <a href="https://luord.com/2022/04/30/domain">domain</a>. In simpler terms: we only care about what our objects can <em>do</em>.
On that note, the code I'll be showing shouldn't throw errors in <a href="https://mypy-lang.org/">mypy</a> or <a href="https://github.com/microsoft/pyright">pyright</a>. <a class="footnote-backref" href="#fnref:protocols" title="Jump back to footnote 1 in the text">↩</a></p>
</li>
<li id="fn:pool">
<p>If we stick to just the parents' genomes, then the target will never be reached if it requires a
gene that neither of the parents has. <a class="footnote-backref" href="#fnref:pool" title="Jump back to footnote 2 in the text">↩</a></p>
</li>
<li id="fn:offspring">
<p>The intermediate class <code>Offspring</code> fulfills two purposes here: to explicitly show
the mutation step (instead of leaving it as an implementation detail of crossover) and to rely
on the type system. We'll know we have a real individual only if it was selected from an
existing population or if it's the result of mutation from the crossover of two parents. <a class="footnote-backref" href="#fnref:offspring" title="Jump back to footnote 3 in the text">↩</a></p>
</li>
<li id="fn:type">
<p>You might have noticed that an "Individual" is represented only by an empty
collection protocol and a generic type (and the type variable might not even be needed after Python 3.12 drops).
This is on purpose: the algorithm doesn't need to care what an individual <em>is</em> beyond "a collection of genes". <a class="footnote-backref" href="#fnref:type" title="Jump back to footnote 4 in the text">↩</a></p>
</li>
<li id="fn:mate">
<p><em>How</em> it finds it is an implementation detail, hopefully one that excludes its parents. <a class="footnote-backref" href="#fnref:mate" title="Jump back to footnote 5 in the text">↩</a></p>
</li>
<li id="fn:jokes">
<p>Since, as we all know, "code is for what, tests are for why, and comments are for jokes". <a class="footnote-backref" href="#fnref:jokes" title="Jump back to footnote 6 in the text">↩</a></p>
</li>
</ol>
</div>Domains of engineers and users2022-04-30T00:00:00-05:002023-12-02T02:09:16-05:00Luis Orduztag:luord.com,2022-04-30:/2022/04/30/domain<p>Improving communication by writing code that describes the domain more closely.</p><p>I finished <a href="https://luord.com/2022/03/31/abstracting">my last post</a> by mentioning how refactoring can help us achieve a code base where discussing
the code and discussing the domain model can be analogous or even entail similar discussions. That's
a topic that deserves a deeper looking into.</p>
<h2>Experts on different domains</h2>
<p>Imagine this conversation:</p>
<blockquote>
<p>End user/Product owner: "We'd like for the URL of the Foo to have the provided identifier instead of these random characters."</p>
<p>Engineer: "Oh, by default the framework adds a UUID as the primary key, which is also used for URLs; we'll update it."</p>
<p>End user/Product owner: "Is that a front-end change or a back-end change?"</p>
</blockquote>
<p>Now, now, that's an exaggeration as I've worked with plenty of users and product people who were well
versed enough on technical details—often because they <em>had</em> to pick up the terminology—to understand what the developer was talking about
in that exchange, but I hope this gets the idea across: Too often, there's a gap in shared terminology that hinders the communication between
engineers and users.</p>
<p>And here's an example in the other direction:</p>
<blockquote>
<p>End user/Product owner: "We need the aggregated results of the 'Foo' to be calculated from the monthly 'bar' instead of the weekly 'bar' from now on."</p>
<p>Engineer: "Understood, I'll add it to the backlog and change it as soon as possible."</p>
<p>Engineer (later, to another engineer or the tech lead): "Where the hell do the aggregated results value come from? Is that in the database? How is it called there? And what even is <code>bar</code>?"</p>
</blockquote>
<p>Now that one is no exaggeration; I've had that exact conversation over the years, multiple times.
And I've been on both sides of that last question too. Of course, I've also been in projects where there was enough rapport between
the engineering and product teams for the engineer to simply ask the product owner right away... But the problem still persists,
only in the form of periodic repetitive conversations instead of the latency caused by the developer looking for someone who understands.</p>
<p>The point is that engineers are experts on the domain of software engineering, while users
are experts on <em>their</em> own domain, which they're hoping it gets easier to do with the
application/platform/system they bought/subscribed to, or hired the engineers to create/maintain.</p>
<h2>Code and stories</h2>
<p>Let's imagine that the user/product owner comes at us with the following requirement:</p>
<blockquote>
<p>As a user, I want to store my "Foo" with the "jon", "doe",
"bar" and "baz" values. With the understanding that "baz" is seven times
"bar".</p>
</blockquote>
<p>After the appropriate back and forth, we come to the following <a href="https://luord.com/2020/02/01/stories">user story</a>:</p>
<dl>
<dt>Store Foo</dt>
<dd>Create a new model in <code>cool_framework</code> used to store the periodic <code>Foo</code> of the
user. The fields are <code>jon</code> (unique string), <code>doe</code> (integer, 100 by default),
and <code>bar</code> (float, cannot be empty); <code>baz</code> will be dynamically generated from <code>bar</code>.
A UUID field will be included as primary key by the framework.</dd>
</dl>
<p>This user story is of course lacking a way for users to actually submit the data they want to store<sup id="fnref:form"><a class="footnote-ref" href="#fn:form">1</a></sup>,
but it already has more than enough to show how the mismatch in communication starts: The user/product owner
could very well ask "what the hell is a UUID?"<sup id="fnref:float"><a class="footnote-ref" href="#fn:float">2</a></sup></p>
<p>I should point out that I am in no way condemning this approach. It's perfectly fine and how around ninety
percent of the projects I've worked on have looked.</p>
<p>But now let's look at the code that fulfills the user story:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">cool_framework</span> <span class="kn">import</span> <span class="n">models</span>
<span class="k">class</span> <span class="nc">Foo</span><span class="p">(</span><span class="n">models</span><span class="o">.</span><span class="n">Model</span><span class="p">):</span>
<span class="n">jon</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">StringField</span><span class="p">(</span><span class="n">unique</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">doe</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">IntegerField</span><span class="p">(</span><span class="n">default</span><span class="o">=</span><span class="mi">100</span><span class="p">)</span>
<span class="n">bar</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">FloatField</span><span class="p">(</span><span class="n">nullable</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
<span class="nd">@models</span><span class="o">.</span><span class="n">computed_property</span>
<span class="k">def</span> <span class="nf">baz</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">bar</span> <span class="o">*</span> <span class="mi">7</span>
</code></pre></div>
<p>Again, this is perfectly fine; I currently work in projects whose code looks like this (with a
billion more fields and methods, of course), and there aren't problems, usually.</p>
<p>The thing is while this code can trivially be discused among engineers, it can't be discussed
by or with users. By this point, both groups are essentially speaking different languages: The users
talk about whatever Foo is, how the "jon" group performed this week, when the "baz" could change, etc.;
the engineers, conversely, are talking about fields, properties, tables, migrations, etc.</p>
<p>And this works! Most of my experience has had this kind of separation and the teams work and the product
is delivered. But what if the language gap could be narrowed?</p>
<h2>A shared language</h2>
<p>Let's rephrase that story:</p>
<dl>
<dt>Store Foo</dt>
<dd>Create a class/type/struct<sup id="fnref:class"><a class="footnote-ref" href="#fn:class">3</a></sup> <code>Foo</code> where the user can store multiple instances of the following
values: <code>jon</code> (type <code>ProvidedIdentifier</code>), <code>doe</code> (type <code>UserEstimation</code>), and <code>bar</code> (type <code>PeriodicResults</code>);
<code>baz</code> (type <code>AggregatedResults</code>) is calculated from <code>bar</code>.</dd>
</dl>
<p>I just made up those types, but what's important here is that we're to assume that those types <em>mean something</em> to
the users. Whatever is in "jon", the users usually call it the "provided identifier" of whatever Foo is; they know
that "bar" is the "periodic results" of whatever it is that the users do. Ditto for the other fields.</p>
<p>With that understanding, let's rewrite the code to start illustrating why representing the domain more explicitly is important:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">decimal</span> <span class="kn">import</span> <span class="n">Decimal</span>
<span class="k">class</span> <span class="nc">ProvidedIdentifier</span><span class="p">(</span><span class="n">string</span><span class="p">):</span>
<span class="k">class</span> <span class="nc">Meta</span><span class="p">:</span>
<span class="n">unique</span> <span class="o">=</span> <span class="kc">True</span>
<span class="k">class</span> <span class="nc">UserEstimation</span><span class="p">(</span><span class="nb">int</span><span class="p">):</span>
<span class="k">class</span> <span class="nc">Meta</span><span class="p">:</span>
<span class="n">default</span> <span class="o">=</span> <span class="mi">100</span>
<span class="k">class</span> <span class="nc">PeriodicResults</span><span class="p">(</span><span class="n">Decimal</span><span class="p">):</span>
<span class="k">class</span> <span class="nc">Meta</span><span class="p">:</span>
<span class="n">nullable</span> <span class="o">=</span> <span class="kc">False</span>
<span class="k">class</span> <span class="nc">AggregatedResults</span><span class="p">(</span><span class="n">Decimal</span><span class="p">):</span>
<span class="n">FACTOR</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">7</span>
<span class="k">def</span> <span class="fm">__new__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">pr</span><span class="p">:</span> <span class="n">PeriodicResults</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__new__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">pr</span> <span class="o">*</span> <span class="bp">self</span><span class="o">.</span><span class="n">FACTOR</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">Foo</span><span class="p">:</span>
<span class="n">jon</span><span class="p">:</span> <span class="n">ProvidedIdentifier</span>
<span class="n">doe</span><span class="p">:</span> <span class="n">UserEstimation</span>
<span class="n">bar</span><span class="p">:</span> <span class="n">PeriodicResults</span>
<span class="nd">@property</span>
<span class="k">def</span> <span class="nf">baz</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">AggregatedResults</span><span class="p">:</span>
<span class="k">return</span> <span class="n">AggregatedResults</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">bar</span><span class="p">)</span>
</code></pre></div>
<p>Is the code more verbose? In total, absolutely<sup id="fnref:lesscode"><a class="footnote-ref" href="#fn:lesscode">4</a></sup> but it too became clearer
about the <em>what</em> we're dealing with: We can assume that the signatures of those classes came from discussions with the users,
where they described what that data means in their business and how it's supposed to behave.
Sure, <code>doe</code> is still a string, <code>bar</code> a floating point value, etcetera, but now the developers can talk about
the code in terms similar to what the users talk about, which also means they'll talk to the <em>users</em> in those terms too.</p>
<p>Let's reimage the conversations from the start, but now with the assumption that all the code is like this:</p>
<p>Well, for starters the first conversation no longer makes sense as another win from these changes is that we decoupled
our business logic from whatever framework we're using<sup id="fnref:clean"><a class="footnote-ref" href="#fn:clean">5</a></sup>... As long as the framework treats the business code as source of truth anyway.</p>
<p>As for the second,</p>
<blockquote>
<p>End user/Product owner: "We need the aggregated results of the 'Foo' to be calculated from the monthly 'bar' instead of the weekly 'bar' from now on."</p>
<p>Engineer: "Just to make sure, <code>baz</code> is the aggregated results and <code>bar</code> is the periodic results, correct?"</p>
<p>End user/Product owner: "Correct!"</p>
<p>Engineer: "Cool, I'll get it done."</p>
</blockquote>
<p>Let's be real, it's likely that the engineer still doesn't know what the hell these "results" are or why the users care about them, but now at least it might be easier
to pinpoint what needs to change without having to find someone who might be more familiar with the domain.</p>
<p>Finally, it bears mentioning that this approach doesn't solve all communication problems: The engineers will never be domain experts on the users' domain, so there will
always be questions, specially when creating new features as the developers will need to ask what the new classes/types will need for fields and metadata<sup id="fnref:terminology"><a class="footnote-ref" href="#fn:terminology">6</a></sup>;
as for the users/product owners, the decoupling with the framework and the increasing reliance on native language constructs means that the number of technical details they need
to have an idea about becomes smaller, but not zero.</p>
<p>Regardless, giving the team (both involved users and developers) a way to more fluidly discuss the product is a huge step forward in my book.</p>
<h2>A caveat</h2>
<p>All of that is pretty nice; however, the boilerplate the users don't care about still needs to exist <em>somewhere</em>. Maybe abstracted and hidden away in specific modules or even
internal libraries that have their own repos, but finding and correcting leaky abstractions is a neverending battle, so those
discussions that ideally should be among engineers only might find their way in conversations with the users.</p>
<p>Imagine a world where we didn't need to abstract the boilerplate for every project; that it simply didn't exist.
In that ideal world, we could just drop some native classes/structs/etc that contain all the business logic and <em>only</em> the business logic to some tool(s),
and the tool would automagically take care of all the wiring needed for the application to reach the end user.</p>
<p>I've given a lot of thought to such an idea, and maybe it's a pipe dream, but stranger things happen in this industry all the time.</p>
<div class="footnote">
<hr>
<ol>
<li id="fn:form">
<p>Let's say a form in some web site/application. <a class="footnote-backref" href="#fnref:form" title="Jump back to footnote 1 in the text">↩</a></p>
</li>
<li id="fn:float">
<p>Or, for that matter "what are strings and floats?" You never know! <a class="footnote-backref" href="#fnref:float" title="Jump back to footnote 2 in the text">↩</a></p>
</li>
<li id="fn:class">
<p>Or whatever construct your favorite language uses to group data, if any! <a class="footnote-backref" href="#fnref:class" title="Jump back to footnote 3 in the text">↩</a></p>
</li>
<li id="fn:lesscode">
<p>But hey, the code of the <code>Foo</code> class itself got simpler. <a class="footnote-backref" href="#fnref:lesscode" title="Jump back to footnote 4 in the text">↩</a></p>
</li>
<li id="fn:clean">
<p>Yay for clean architecture! <a class="footnote-backref" href="#fnref:clean" title="Jump back to footnote 5 in the text">↩</a></p>
</li>
<li id="fn:terminology">
<p>In other words, some conversations for the developers to make the business rules clearer, which they then can express in the code! <a class="footnote-backref" href="#fnref:terminology" title="Jump back to footnote 6 in the text">↩</a></p>
</li>
</ol>
</div>Practical refactoring: Abstractions2022-03-31T00:00:00-05:002023-12-02T02:09:16-05:00Luis Orduztag:luord.com,2022-03-31:/2022/03/31/abstracting<p>Improving code by way of writing specific and self-describing abstractions.</p><p>In <a href="https://luord.com/2022/02/28/refactoring">my last post</a>, we did a basic rundown of a very convoluted short algorithm to make
more explicit what was actually happening in it. That by itself goes a long way in improving how readable the code is, and thus makes it easier to maintain.
I've seen small improvements like that be welcome enthusiastically among different teams, but we can go
further.</p>
<p>I remember a project I worked on where there was basically no separation of concerns between request
handling boilerplate, database connection boilerplate and actual business logic; everything was handled
within the same functions. It was a nightmare. I was hired to create some new APIs, but it took me just a
week of trying to create new handlers like that to decide that such was no way to live. I took it
upon myself to refactor that code. It bears reasserting that when refactoring, it's good—and often enough—to first reach for the lowest hanging fruit and in that code the easiest improvement was, of course,
separating code that dealt with different things into different functions, and calling those new functions from the old ones.</p>
<p>Now, the small script we're going through does really only one thing, but that doesn't mean we can't
divide some responsibilities by way of abstracting away some code <strong>not</strong> directly related to that
one thing. If we do this, we improve the code in at least three ways:</p>
<ol>
<li>The algorithm itself becomes more immediately obvious; it's easier to understand what the code does.</li>
<li>By abstracting the supporting/boilerplate code, we make it possible to reuse those same structures
somewhere else.</li>
<li>This gives us a opportunity to bring the code in step with the domain: using names specific to what
we're doing instead of talking exclusively about basic data types.</li>
</ol>
<h2>The abstractions</h2>
<p>Something I didn't mention in my last post is that the small method we're refactoring is actually part of a basic
genetic algorith I implemented for fun. Well, genetic algorithms deal with populations, so what if
instead dealing with "lists" of "strings", we create an actual <code>Population</code> class/data type that does
what we need our populations to do:</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">Population</span><span class="p">(</span><span class="nb">list</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">Iterable</span><span class="p">[</span><span class="n">Self</span><span class="p">]:</span>
<span class="k">return</span> <span class="nb">iter</span><span class="p">(</span><span class="bp">self</span><span class="p">[</span><span class="n">x</span><span class="p">::</span><span class="mi">2</span><span class="p">]</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">2</span><span class="p">))</span>
</code></pre></div>
<p>Not complicated, now we know that a population is a list, but one that when iterated just produces
two sublists: the two halves of the original. We <em>can</em> improve it further, but this is enough for
what we need<sup id="fnref:premature"><a class="footnote-ref" href="#fn:premature">1</a></sup>.</p>
<p>We have our population, but our genetic algorithm is not of random things; it specifically looks for
individuals that match a given root individual, with the purpose of each generation to be closer to that individual.</p>
<p>Let's give it a try:</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">Individual</span><span class="p">(</span><span class="nb">str</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__and__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">other</span><span class="p">:</span> <span class="n">Self</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">sum</span><span class="p">(</span><span class="n">ch_s</span> <span class="o">==</span> <span class="n">ch_o</span> <span class="k">for</span> <span class="n">ch_s</span><span class="p">,</span> <span class="n">ch_o</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">other</span><span class="p">))</span>
</code></pre></div>
<p>For our "model"<sup id="fnref:genetic"><a class="footnote-ref" href="#fn:genetic">2</a></sup> we know that we need a specific type of string that when compared with another will
return the number of identical characters. So we do just that: create a <code>str</code> subclass that
can "intersect" with other strings of the same type, and give a numeric value representing how well
they match.</p>
<p>With these two abstractions alone, our method improves considerably:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">get_best_matches</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">WinnerPair</span><span class="p">:</span>
<span class="c1"># self.population: Population[Individual]</span>
<span class="c1"># self.root: Individual</span>
<span class="n">winners</span> <span class="o">=</span> <span class="n">WinnerPair</span><span class="p">()</span>
<span class="k">for</span> <span class="n">half</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">population</span><span class="p">:</span>
<span class="n">winner</span> <span class="o">=</span> <span class="nb">max</span><span class="p">(</span><span class="n">half</span><span class="p">,</span> <span class="n">key</span><span class="o">=</span><span class="k">lambda</span> <span class="n">ind</span><span class="p">:</span> <span class="n">ind</span> <span class="o">&</span> <span class="bp">self</span><span class="o">.</span><span class="n">root</span><span class="p">)</span>
<span class="n">winners</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">winner</span><span class="p">)</span>
<span class="k">return</span> <span class="n">winners</span>
</code></pre></div>
<p>I <em>like</em> it. We get each member of the winner pair from half of the population, and notice how the code
doesn't show things we don't need to know to understand the steps:</p>
<ol>
<li>How do we get each half of the population? That doesn't matter for understanding the algorithm; we just
need to know that we're getting a half. How that half is gathered is up to the implementtion in <code>Population</code>
which we can check if we need to, or we could change it if we have to: Like instead of appending the
odd-positioned items to a half and the even-positioned to the other half, we could just literally split it
at the central index.</li>
<li>To get the winner we're comparing how well is the intersection between each individual and the root
(<code>self.root</code>, renamed from <code>self.word</code><sup id="fnref:word"><a class="footnote-ref" href="#fn:word">3</a></sup>, in this case). How is that intersection calculated/found?
That's entirely an implementation detail, which again we can check and/or change in <code>Individual</code>
if we have to. We could even make individuals a different type instead of strings, make a comparison
appropriate for that type, and we wouldn't need to change <strong>anything</strong> in this method.</li>
</ol>
<p>Notice that, at the end of my previous post, I said that we could use abstractions instead of relying
on comments to tell us what the data types are supposed to represent. There are still comments in this
piece of code... but that comment is just to note <em>what</em> abstractions we're using, and is there just
for description purposes. In the full code it would be completely redundant, since the types of
<code>self.population</code> and <code>self.root</code> would already be defined somewhere else. Likely at the top of the class
this method belongs to.</p>
<p>Nonetheless, we could still make those comments explicit in the code<sup id="fnref:jokes"><a class="footnote-ref" href="#fn:jokes">4</a></sup> by way of, say, making them
arguments to the method. However, <em>that</em> would no longer be a refactoring<sup id="fnref:zero"><a class="footnote-ref" href="#fn:zero">5</a></sup>... But we can cheat a little
bit:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">get_winners</span><span class="p">(</span>
<span class="n">population</span><span class="p">:</span> <span class="n">Population</span><span class="p">[</span><span class="n">Individual</span><span class="p">],</span> <span class="n">root</span><span class="p">:</span> <span class="n">Individual</span>
<span class="p">)</span> <span class="o">-></span> <span class="n">WinnerPair</span><span class="p">:</span>
<span class="n">winners</span> <span class="o">=</span> <span class="n">WinnerPair</span><span class="p">()</span>
<span class="k">for</span> <span class="n">half</span> <span class="ow">in</span> <span class="n">population</span><span class="p">:</span>
<span class="n">winner</span> <span class="o">=</span> <span class="nb">max</span><span class="p">(</span><span class="n">half</span><span class="p">,</span> <span class="n">key</span><span class="o">=</span><span class="k">lambda</span> <span class="n">ind</span><span class="p">:</span> <span class="n">ind</span> <span class="o">&</span> <span class="n">root</span><span class="p">)</span>
<span class="n">winners</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">winner</span><span class="p">)</span>
<span class="k">return</span> <span class="n">winners</span>
<span class="k">def</span> <span class="nf">get_best_matches</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">WinnerPair</span><span class="p">:</span>
<span class="k">return</span> <span class="n">get_winners</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">population</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">root</span><span class="p">)</span>
</code></pre></div>
<p>Basically, we create a new <em>function</em> that isn't necessarily tied to a class (note the lack of <code>self</code>) and we
call that function from our method. This gives us the potential advantage of reusing that very same winner finding
logic somewhere else if the opportunity arises.</p>
<h2>Hindsight</h2>
<p>So there you have it, we improved the code, considerably, just by moving a few lines around and making
what we're doing and using more explicit. It goes without saying that this would help us communicate between the developers <em>and</em> with
the product team (if any) much more easily.</p>
<p>Ideally, in a good codebase, discussing the code and discussing the
business model would entail very similar expressions, as the code would just be a literal (with caveats of course) representation
of that business model. Hopefully this post showed how such a thing could be achieved.</p>
<p>This is a subject I really, really like and I'm hoping to keep writing about it.</p>
<div class="footnote">
<hr>
<ol>
<li id="fn:premature">
<p>When refactoring, it's easy to get lost making increasingly minute improvements. We always
gotta remember that premature optimization is the root of all evil and that abstractions can easily
become unnecessary indirections. This is a fine line, and sometimes I don't spot it, so it's important to
remember that the goal is to make the code <em>more</em> maintainable instead of perfect from the get go (which is impossible anyway). In an
actual product, it also helps to remember what I think as the rule zero of Software Engineering:
"fight for the users". If something we're doing won't improve the users'
experience in any meaningful way, it might be best left alone. <a class="footnote-backref" href="#fnref:premature" title="Jump back to footnote 1 in the text">↩</a></p>
</li>
<li id="fn:genetic">
<p>This genetic algorithm in particular just iterates over populations that increasingly
resemble the root word. <a class="footnote-backref" href="#fnref:genetic" title="Jump back to footnote 2 in the text">↩</a></p>
</li>
<li id="fn:word">
<p>In my opinion, the naming of the root individual as <code>self.word</code> was a code smell born out of
the usage of basic data types instead of proper abstractions. It was a way to hint at the developer that
the comparison was between strings. Using proper types/classes, we no longer need to do that. <a class="footnote-backref" href="#fnref:word" title="Jump back to footnote 3 in the text">↩</a></p>
</li>
<li id="fn:jokes">
<p>After all, "code is for what, tests are for why and comments are for jokes" (which is also a
joke... or is it?) <a class="footnote-backref" href="#fnref:jokes" title="Jump back to footnote 4 in the text">↩</a></p>
</li>
<li id="fn:zero">
<p>If "improve the easiest thing first" is rule one of refactoring, then "do <strong><em>not</em></strong>, under any
circumstance, change the contract" is rule zero of refactoring. To put it another way: if we have a good
test suite, a proper refactoring shouldn't change the result of <em>any</em> test. Anything other than that is
an actual change in the behavior of the application/system, and it better have been agreed upon. <a class="footnote-backref" href="#fnref:zero" title="Jump back to footnote 5 in the text">↩</a></p>
</li>
</ol>
</div>Practical refactoring: 'clever' code2022-02-28T00:00:00-05:002023-12-02T02:09:16-05:00Luis Orduztag:luord.com,2022-02-28:/2022/02/28/refactoring<p>Practical example of the problems with clever code and the benefits of refactoring.</p><p>Look at this code</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">get_best_matches</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">WinnerPair</span><span class="p">:</span>
<span class="w"> </span><span class="sd">"""</span>
<span class="sd"> Divide population in half.</span>
<span class="sd"> Pick the word closest to the matching word in each half.</span>
<span class="sd"> """</span>
<span class="k">return</span> <span class="n">WinnerPair</span><span class="p">(</span><span class="o">*</span><span class="nb">map</span><span class="p">(</span>
<span class="k">lambda</span> <span class="n">population_half</span><span class="p">:</span> <span class="nb">max</span><span class="p">(</span>
<span class="n">population_half</span><span class="p">,</span>
<span class="n">default</span><span class="o">=</span><span class="s2">""</span><span class="p">,</span>
<span class="n">key</span><span class="o">=</span><span class="k">lambda</span> <span class="n">word</span><span class="p">:</span> <span class="nb">sum</span><span class="p">(</span>
<span class="n">a</span> <span class="o">==</span> <span class="n">b</span> <span class="k">for</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">word</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">word</span><span class="p">)</span>
<span class="p">)</span>
<span class="p">),</span>
<span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">population</span><span class="p">[::</span><span class="mi">2</span><span class="p">],</span> <span class="bp">self</span><span class="o">.</span><span class="n">population</span><span class="p">[</span><span class="mi">1</span><span class="p">::</span><span class="mi">2</span><span class="p">])</span>
<span class="p">))</span>
</code></pre></div>
<p>I'm not gonna deny it, I liked writing it, I like that it is technically a single function call<sup id="fnref:class"><a class="footnote-ref" href="#fn:class">1</a></sup>,
the usage of lambdas and the built-in Python functions used for handling, well, functions and iterables.
What can I say? It makes me feel "clever" because technically it's code that requires certain level of
familiarity with the language.</p>
<p>It's also a total mess. I literally spent an entire afternoon explaining this "short" piece of code
to an experienced engineer who had already invested a few months getting familiar with Python.</p>
<p>This code is me at my most self-indulgent and I'm well aware I would never have written this outside
of a <a href="https://gitlab.com/luord/prototype">prototype meant only for me to play around</a>. Code like this is <strong>not</strong> meant to live in a
system worked at by more than one developer. It'd be a nightmare to maintain, as only the one who wrote
it could possibly understand it. Hell, I wrote this and I had to struggle a bit puzzling what it actually
did.</p>
<p>In short, this code is ripe for improvement, which is exactly what I'm gonna do.</p>
<h2>Unclear iterations</h2>
<p>The first thing that jumps at me upon seeing this code is that there are three nested iterations in it,
but it's very difficult to tell which one is which or where each one ends.
An easy first fix is then relying less on built-in functions and making the iterations more explicit
via <code>for</code> statements.</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">get_best_matches</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">WinnerPair</span><span class="p">:</span>
<span class="c1"># Get the words closest to the target in each half of the population</span>
<span class="n">winners</span> <span class="o">=</span> <span class="n">WinnerPair</span><span class="p">()</span>
<span class="k">for</span> <span class="n">population</span> <span class="ow">in</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">population</span><span class="p">[</span><span class="n">x</span><span class="p">::</span><span class="mi">2</span><span class="p">]</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">2</span><span class="p">)):</span>
<span class="n">scores</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">word</span> <span class="ow">in</span> <span class="n">population</span><span class="p">:</span>
<span class="n">similarity</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">for</span> <span class="n">char_word</span><span class="p">,</span> <span class="n">char_target</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">word</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">word</span><span class="p">):</span>
<span class="n">similarity</span> <span class="o">+=</span> <span class="n">chard_word</span> <span class="o">==</span> <span class="n">char_target</span>
<span class="n">scores</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">word</span><span class="p">,</span> <span class="n">score</span><span class="p">))</span>
<span class="n">winner</span> <span class="o">=</span> <span class="nb">max</span><span class="p">(</span><span class="n">scores</span><span class="p">,</span> <span class="n">key</span><span class="o">=</span><span class="k">lambda</span> <span class="n">score</span><span class="p">:</span> <span class="n">score</span><span class="p">[</span><span class="mi">1</span><span class="p">])</span>
<span class="n">winners</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">winner</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
<span class="k">return</span> <span class="n">winners</span>
</code></pre></div>
<p>Looks quite different, doesn't it? It would seem to someone completely unfamiliar with Python that
I changed more than replacing the function calls with <code>for</code>s, but that's truly <em>all</em> I did:</p>
<ol>
<li>The first function was <code>map</code> which is doing <em>something</em> to both halves of the population.</li>
<li>The second function was <code>max</code> which is picking the highest according to <em>something</em> in each word in
the population.</li>
<li>The third function was <code>sum</code> which is actually calculating that previous "something": In this case,
how similar is the current word with the target word.</li>
</ol>
<p>I then reused <code>max</code>, but it's now clearer what maximum value of what it's being picked. I will not lie:
I hesitated with leaving the <code>sum</code> as it was as I felt that with the other replacements it was clear enough,
but then I saw the opportunity to further clarify that we were comparing the current word with the target word.
On the other hand, I did leave <code>zip</code> as it was, as that one <strong>is</strong> clear enough to me.<sup id="fnref:craft"><a class="footnote-ref" href="#fn:craft">2</a></sup></p>
<hr>
<p>Aside: Someone with some familiarity with algorithm analysis might see three nested <code>for</code> loops and
pale at the "cubic" complexity, but this function isn't iterating over the population input
(let's call it "n") multiple times. It's instead iterating only once over the <em>total characters</em> input
(let's say "m"). In short: This iteration only visits each character in the population once.</p>
<p>The word list (but not each word) <em>is</em> visited twice because of the <code>max</code> function, but since two is a
constant, it remains of linear complexity.</p>
<hr>
<p>Anyhow, those "straightfoward" changes are enough to at least being able to tell what the function
is doing line by line, but it can be better.</p>
<p>As it is, we're doing a bunch of operations over basic data types with a comment explaining what
those data types are supposed to represent. We could instead explicitly define our own abstractions over
those data types and let those abstractions tell us what they can or can't do, or how they should be
used.</p>
<p>But I feel like that is interesting enough for its own post, so see you <a href="https://luord.com/2022/03/31/abstracting">in the next part</a>!</p>
<div class="footnote">
<hr>
<ol>
<li id="fn:class">
<p>Well, a function call wrapped in a class instantiation, but who's nitpicking? <a class="footnote-backref" href="#fnref:class" title="Jump back to footnote 1 in the text">↩</a></p>
</li>
<li id="fn:craft">
<p>I firmly think that Software Engineering <strong>is</strong> engineering, and I have no problem calling myself "engineer" over, say, "craftsman", but there <em>is</em> a subjective factor to some decisions. <a class="footnote-backref" href="#fnref:craft" title="Jump back to footnote 2 in the text">↩</a></p>
</li>
</ol>
</div>21 & 222022-01-01T00:00:00-05:002023-12-02T02:09:16-05:00Luis Orduztag:luord.com,2022-01-01:/2022/01/01/2021<p>Summary of my 2021, some ideas for 2022</p><h2>2021</h2>
<p>I wish I could say 2021 was an eventful year, but frankly I spent most of it dealing with ennui, boredom and general disenchantment.
One could even say I might have faced some burnout, but I don't think that's it; I still very much like writing code.</p>
<p>I also got covid right in the middle of the year so that might also have put a damper on things, but that doesn't mean that absolutely
nothing happened last year.</p>
<h3>Life</h3>
<p>Back in June 2020, and rather suddenly, an opportunity to accomplish one of my personal goals presented itself, and I decided to go
for it with the caveat that I had to complete it within ten months. Indeed, I finished it by April, and now that's one fewer thing
for me to worry about, forever.</p>
<p>I moved back to Bucaramanga, my home city, after spending three years in Bogotá.
I figured I wasn't doing much of anything there (even less with covid around) so I decided I wanted to spend more time with my folks,
siblings and my sister's children.</p>
<p>I could finally go back to the movies after over a year of lockdown, but more on that later.</p>
<h3>Work</h3>
<p>I started working at a new company, with which I'm extremely happy. It fits all I want from a place to work nigh perfectly
(remote, flexible, well organized, interest in new tech, etc) and the team is great.
Some of the projects are better than others but regardless, it would take a lot, <em>a lot</em>, for me to leave.</p>
<p>On the personal projects front, I created a prototype to compare single page applications with backend generated fragments displayed
by <a href="https://htmx.org/">htmx</a>.<sup id="fnref:htmx"><a class="footnote-ref" href="#fn:htmx">1</a></sup> Also I learned a little about genetic algorithms while on it.</p>
<p>Did some updates to this website, mainly related to <a href="https://indieweb.org/">indie web</a> things.</p>
<h3>Reading</h3>
<h4>Technical</h4>
<p>I cut back on my HN surfing. Not because the signal to noise ratio has gotten lower (although I've gotten the feeling sometimes)
but because I was simply spending too much time in, often repetitive, comment sections. And I didn't even comment, I mostly read.</p>
<p>Instead, I've been subscribing to more and more blogs (both company tech blogs and engineers' personal blogs), to the point that
my subscription list in <a href="https://theoldreader.com/">the old reader</a> tripled over the year. I'd say the change is good in general.</p>
<p>Likewise, I've followed more people on social media (primarily the same engineers whose blogs I subscribed to),
mostly for discovery value: Often those people share stuff by other people, which means potentially new blogs to follow.<sup id="fnref:sm"><a class="footnote-ref" href="#fn:sm">2</a></sup></p>
<h4>Fiction</h4>
<p>I finally got an e-reader last year, and it <em>has</em> translated on my reading quite a few more books than in recent years.
Mostly fantasy, and I'd say my favorite was probably "<a href="https://www.goodreads.com/book/show/9579634-prince-of-thorns">Prince of Thorns</a>".
I'll be reading its sequels and more this year.</p>
<h3>Movies</h3>
<p>According to <a href="https://www.criticker.com/profile/luord/">my Criticker profile</a>, I watched sixty six films last year.
I love that almost half of them were watched in theaters, although I <em>do</em> stream movies; had to pick up the habit during 2020.</p>
<p>Apparently, the movie I hated the most overall was some cliched horror movie from 2019, but the one I hated the most that I watched
in theaters was of course the new "matrix" movie.</p>
<p>Conversely, I gave the exact same rating, the highest of the 66, to two different movies: "Savig private Ryan",
which I watched at home, and "Spider-Man: No way home", which I watched in theaters. Yup, I stand by it,
definitely the two movies I liked the most that I saw last year.</p>
<p>Unfortunately, nothing over 85 (which would translate to a nine or higher in <a href="https://www.imdb.com/user/ur39224109">imdb</a>),
and indeed, no movie I watched last year really floored me.
I'm hopeful for "<a href="https://www.imdb.com/title/tt11138512/">The Northman</a>" to pull it off this 2022.</p>
<h3>Music</h3>
<p>I don't stream music; I'd rather buy the songs I like and listen only to my collection... But that didn't happen often last year.
Though I did remember adding "<a href="https://www.youtube.com/watch?v=ocpDEOXABWg">Courtesy Call</a>" as well as some instrumental tracks to
my "music for programming/writing" playlist.</p>
<h2>2022</h2>
<p>With 2021 out of the way, I do have plans for this year.</p>
<h3>Life</h3>
<p>I'm pretty happy with where my life is right now, materially speaking, so I'm focusing this year on finally losing weight.
I have a plan this time.</p>
<h3>Work</h3>
<p>Will keep working in my current job. As I said, I have zero interest in leaving my current company.</p>
<p>Will do far more experimentation, both to practice stuff I might have gotten rusty with and to get to know new things... such as Rust.</p>
<p>I also have an idea for a potential tool/service that could become my first officially released open source project. Gotta
work on it and iterate to see if it actually is viable or not.</p>
<h3>Reading</h3>
<p>Nothing much, will continue the trend of reading the blogs I already follow and subscribe to more.</p>
<p>Likewise, more usage of my e-reader for fiction.</p>
<h3>Writing</h3>
<p>I'll do my best to be more active on this blog. Can't promise a regular schedule to myself, but I will write at least a
<a href="/category/notes">note</a> of anything that comes to mind or that I find interesting.</p>
<p>Will finish some stories I am writing privately and will try to get started on something I feel I can actually publish. To get
feedback if nothing else.</p>
<h3>Movies</h3>
<p>As I said, I'm really looking forward to "The Northman", but also there's a handful of other movies that I'm hoping they're as
great as they look (looking at you "Doctor Strange") or that end up surprising me
(looking at you "Avatar 2" or "The secrets of Dumbledore", among others).</p>
<div class="footnote">
<hr>
<ol>
<li id="fn:htmx">
<p>Long story short: HTMX is absolutely magical and everyone should be using it. <a class="footnote-backref" href="#fnref:htmx" title="Jump back to footnote 1 in the text">↩</a></p>
</li>
<li id="fn:sm">
<p>I still actively avoid the awful, outrage-obsessed side of social media, of course. <a class="footnote-backref" href="#fnref:sm" title="Jump back to footnote 2 in the text">↩</a></p>
</li>
</ol>
</div>Creating stories from requirements2020-02-01T00:00:00-05:002023-12-02T02:09:16-05:00Luis Orduztag:luord.com,2020-02-01:/2020/02/01/stories<p>On crafting actionable development stories from requirements</p><p>A few weeks ago, one of my best friends came to me with an idea for an application. Unlike most (if not all) of my ideas, this one I actually believe to have potential so I asked him to prepare a list of requirements we could use to at least have a rough goal to aim for with this application. He delivered, and I was presented a good enough list... And I did nothing with it, I've been letting it sit there, all unfulfilled potential. But that changes now!</p>
<p>I've been working as a software engineer for a while and that experience, the <a href="https://agilemanifesto.org/">manifesto</a>, what I remember from my college studies and what I've read regarding system design all point to the same conclusion: Having a fixed list of tasks and dedicating too much time to it is pointless because requirements change like waves in the sea.</p>
<p>However, I still feel it's important to have a set of stories to work on, both to have a general idea of what the system is and how it's supposed to work, as well as for documenting progress; nothing kills motivation like the feeling you're not actually advancing towards your goal. Besides, we control the requirements in this case and, even though waves are fickle, <a href="https://oceanservice.noaa.gov/facts/conveyor.html">the global conveyor belt</a> is still a thing. Much like with that <abbr title="sorry">overly elaborate metaphor</abbr>, my goal is not to describe how every single detail in the application is supposed to work, but more a general description of each functionality, and general details on how to implement it.</p>
<p>With that in mind, I'll show how I approached turning that requirements list written in the "as a user" format into stories that developers can actually work on. I'll forgo things like "story points" or "acceptance criteria" because not only are those restrictive, I'm doing this as a developer for developers—all two of us—and what I care about is what to do and how to do it. And that's also the reason I won't be using stuff like certain project management tool that most software development teams are familiar with whose name I won't mention, at the risk of summoning giant lizards.<sup id="fnref:jira"><a class="footnote-ref" href="#fn:jira">1</a></sup></p>
<p>In fact, this is a good point to mention <a href="https://sourcehut.org/">sourcehut</a>, Drew DeVault's cool set of tools for creating software that's very reminiscent of how the biggest open source projects are maintained. I'm going to be creating these stories using sourcehut's todo, which is to say: simple issues.</p>
<p>Without further ado, let's take one of the requirements as example, one that is universal enough that I can use without risking IP: user accounts. This is an abbreviation of my friend's requirement:</p>
<blockquote>
<p>As a user I can create an account in the application, with an email or using social networks (an email should be sent introducing the platform upon registration).</p>
</blockquote>
<p>As mentioned, my goal with the stories is knowing what to do and how to do it, from a developer's perspective. Part by part, this is the architecture<sup id="fnref:arch"><a class="footnote-ref" href="#fn:arch">2</a></sup> that this requirement defines:</p>
<ol>
<li>The <abbr title="excuse me being captain obvious">application has a domain</abbr>, and this domain includes the entity User because that's what we want to create.</li>
<li>The application has some sort of data layer that the application interfaces with to store this User data.</li>
<li>The application has adapters that <em>interface</em> with third parties to retrieve the user data, social networks in this case.</li>
<li>The application has the <strong>use case</strong> "create user" that is called in two different ways: using email or using the adapters mentioned above.</li>
<li>The application has an <em>interface</em> that the user employs to send the email and other data, or to trigger the retrieval from third parties.</li>
<li>The application has a second <strong>use case</strong>, which is to send an automated email to the, well, email provided by the user upon successful creation.</li>
</ol>
<p>With that rough outline, we have an idea for two or three stories, because at this stage is better to restrict stories to the number of use cases, or to the number of times all use cases are instantiated across the application.<sup id="fnref:cases"><a class="footnote-ref" href="#fn:cases">3</a></sup> Of course, stories ultimately can involve <em>editing</em> use cases too; the point is that we should make the stories about the application business rules whenever possible.</p>
<p>The stories mentioned here are deliberately vague on the tech stack because I want this to be applicable for as many developers as possible.</p>
<dl>
<dt>Create User from email</dt>
<dd>Create an Use Case that accepts raw data as well as a data repository<sup id="fnref:postgres"><a class="footnote-ref" href="#fn:postgres">4</a></sup> and creates an instance of the User domain model using the raw data. It then passes this data to the repository for creation.</dd>
<dd>Create an interface adapter<sup id="fnref:flask"><a class="footnote-ref" href="#fn:flask">5</a></sup> that receives data (including email) submitted by an user and passes it to the aforementioned Use Case, alongside the database repository.</dd>
<dd>Create a view<sup id="fnref:web"><a class="footnote-ref" href="#fn:web">6</a></sup> that allows the user to submit this data to the adapter.<sup id="fnref:server"><a class="footnote-ref" href="#fn:server">7</a></sup></dd>
<dt>Create User from third party</dt>
<dd>Create an interface adapter that gathers data (including email) about an user sent from third parties<sup id="fnref:dance"><a class="footnote-ref" href="#fn:dance">8</a></sup>. It passes this data to the create user Use Case alongside the database repository.<sup id="fnref:clean"><a class="footnote-ref" href="#fn:clean">9</a></sup></dd>
<dd>Create a view that allows the user to go through the third party communication cycle.</dd>
<dt>Send email on user creation</dt>
<dd>Edit Create User Use case so that, upon successful storage, calls a new Use Case.</dd>
<dd>This new Use Case is in charge of sending the body and addressee of an email to an email sender interface adapter.<sup id="fnref:flow"><a class="footnote-ref" href="#fn:flow">10</a></sup></dd>
<dd>Create an interface adapter that uses the data produced by the use case and sends it to an external queue<sup id="fnref:celery"><a class="footnote-ref" href="#fn:celery">11</a></sup>, in the form of a message, that should send the email.</dd>
</dl>
<p>These stories are somewhat vague, and that is deliberate. I don't want to restrict—and not only because I would be restricting myself—and details of implementation are what code reviews and tests are for. The one exception I make on not specifying implementation is regarding the tech stack itself: it's such a big decision that all developers are benefitted if the stack is clear<sup id="fnref:caveat"><a class="footnote-ref" href="#fn:caveat">12</a></sup>.</p>
<p>The stories should only define a rough end goal, and I believe the ones I wrote here achieve that. Anyhow, I feel that this is a good first step and a general description of how we'll be working on this project.</p>
<p>This is going to be part of a series, an idea born from a <a href="https://flaviocopes.com/blog-seo/">great article on blogging</a> that I read the other day, which inspired me to write more and gave me the idea of how to actually write: I'm going to push myself into working on this application (and refactoring an older one, with another friend) so that I have material to write for this blog, and I can use the will to write for the blog as incentive to work on those projects, killing two birds with one stone.</p>
<p><strong>IMPORTANT</strong>: Everything I'm writing in these series is my interpretation and general idea of a good process. Anything (or everything) I write might be entirely wrong and, in such event, I <abbr title="read: beg">encourage</abbr> you to correct me in the comments.</p>
<div class="footnote">
<hr>
<ol>
<li id="fn:jira">
<p>Just FYI, I don't hate it, it can be a great tool, as long as it isn't drowned in the bastardization of scrum, which I don't hate either... as long as it's used as a guideline instead of a forced two-week waterfall grind. <a class="footnote-backref" href="#fnref:jira" title="Jump back to footnote 1 in the text">↩</a></p>
</li>
<li id="fn:arch">
<p>I'm trying to describe this within the terms of the <a href="https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html">clean architecture</a>. Results may vary. <a class="footnote-backref" href="#fnref:arch" title="Jump back to footnote 2 in the text">↩</a></p>
</li>
<li id="fn:cases">
<p>And debatably at any stage. I've seen projects where stories are created for everything, even changing the color of a button to a slightly differen shade of blue. YMMV on advantages and disadvantages of that practice. <a class="footnote-backref" href="#fnref:cases" title="Jump back to footnote 3 in the text">↩</a></p>
</li>
<li id="fn:postgres">
<p>I'm a postgreSQL user. <a class="footnote-backref" href="#fnref:postgres" title="Jump back to footnote 4 in the text">↩</a></p>
</li>
<li id="fn:flask">
<p>Probably a function that would retrieve the data in a given format, create a domain entity object from it that would then pass this object into the database adapter, probably a SQLAlchemy model. This function itself would be called from a <a href="https://www.palletsprojects.com/p/flask/">flask</a> endpoint. <a class="footnote-backref" href="#fnref:flask" title="Jump back to footnote 5 in the text">↩</a></p>
</li>
<li id="fn:web">
<p>I've worked as a full stack web developer almost exclusively, so in here I'm thinking of a form, created either from a JS framework or just a simple web form. <a class="footnote-backref" href="#fnref:web" title="Jump back to footnote 6 in the text">↩</a></p>
</li>
<li id="fn:server">
<p>Of course, nobody is forced to use a client-server system, but it's what I'll use. <a class="footnote-backref" href="#fnref:server" title="Jump back to footnote 7 in the text">↩</a></p>
</li>
<li id="fn:dance">
<p>Probably something like <a href="https://flask-dance.readthedocs.io/en/latest/">flask-dance</a>. <a class="footnote-backref" href="#fnref:dance" title="Jump back to footnote 8 in the text">↩</a></p>
</li>
<li id="fn:clean">
<p>Remember not to pollute the business rules or the entities with data or steps specific to any third party. <a class="footnote-backref" href="#fnref:clean" title="Jump back to footnote 9 in the text">↩</a></p>
</li>
<li id="fn:flow">
<p>Remember not to break flow of control, these business rules don't care how the database signals success or how the sender adapter sends the message. But that's an implementation detail that should be discussed in review. <a class="footnote-backref" href="#fnref:flow" title="Jump back to footnote 10 in the text">↩</a></p>
</li>
<li id="fn:celery">
<p>Probably rabbitmq through Celery. <a class="footnote-backref" href="#fnref:celery" title="Jump back to footnote 11 in the text">↩</a></p>
</li>
<li id="fn:caveat">
<p>Everything beyond that (meaning <em>how</em> the stack is used) is outside the scope of the story. That's for tests and reviews. <a class="footnote-backref" href="#fnref:caveat" title="Jump back to footnote 12 in the text">↩</a></p>
</li>
</ol>
</div>Oracle in Docker2017-11-13T00:00:00-05:002023-12-02T02:09:16-05:00Luis Orduztag:luord.com,2017-11-13:/2017/11/13/oracle<p>Using the official oracle docker images in development</p><p>A while ago, I had to work in a project that used oracle as its data layer (yeah, I know...).
When we started, there was no such thing as an Oracle docker image so the development environment
was either set-up manually or using bash scripts. I tried to create images but, first, it was hell and, second, I didn't want to
bother with any license breach. I love bash and I'm often scripting away repetitive stuff
but I am way too used to docker for my development environments (and also for deploying and in production);
as such, it can be said that, whenever I had to rebuild the environment from scratch (and since a migration was being
made towards data warehouses, that was more often than usual), I cursed my days.</p>
<p>Thankfully, by the time we were finishing and regressions were becoming more and more expected, <a href="https://www.oracle.com/corporate/pressrelease/docker-oracle-041917.html">Oracle released
official images to the docker store</a>. I didn't waste time and, with some effort as the documentation
was quite sparse, I managed to set them up locally and turned the bash scripts (and some plain
text instructions) and other requirements into a <code>docker-compose</code> file. This short guide is about duplicating
the process (well, the Oracle part).</p>
<h2>Initial steps</h2>
<p>First of all, create an account in the <a href="https://store.docker.com/">docker store</a> if you don't have one already.</p>
<p>Next, login with your account in the docker console, using the command <code>docker login</code>.</p>
<h2>Getting Oracle</h2>
<p>With that set up, head over to the <a href="https://hub.docker.com/_/oracle-database-enterprise-edition">oracle enterprise page</a> in the docker store and click in the
button that says "<em>Proceed to Checkout</em>".</p>
<p>At this point, fill the information requested and accept the terms, the process is similar to the one Oracle has for downloading the
client and databases from their website. They require it here too because this is Oracle.</p>
<p><em>Now</em> you can pull the docker image: <code>docker pull store/oracle/database-enterprise:12.2.0.1</code>. It'll take a while.</p>
<h2>Using Oracle</h2>
<p>At this point you're probably in the instructions page, which is now far more detailed than it was when the images were released,
lucky you. They are relatively easy to follow but I'll write the last few commands required to use the image here anyway.</p>
<p>To start the image, run the command:</p>
<div class="highlight"><pre><span></span><code>docker<span class="w"> </span>run<span class="w"> </span>-d<span class="w"> </span>--name<span class="w"> </span><db-container-name><span class="w"> </span><span class="se">\</span>
<span class="w"> </span>store/oracle/database-enterprise:12.2.0.1
</code></pre></div>
<p>To connect to the database using Oracle's sqlplus client, use the following command:</p>
<div class="highlight"><pre><span></span><code>docker<span class="w"> </span><span class="nb">exec</span><span class="w"> </span>-it<span class="w"> </span><db-container-name><span class="w"> </span>bash<span class="w"> </span>-c<span class="w"> </span><span class="se">\</span>
<span class="w"> </span><span class="s2">"source /home/oracle/.bashrc; sqlplus /nolog"</span>
</code></pre></div>
<h2>Some options</h2>
<ul>
<li>Setting the <code>DB_SID</code> environment variable changes the name of the database. Default is <code>ORCLCDB</code>.</li>
<li>The port <code>1521</code> can be mapped so that the container can be accessed from the host. It can also, of course, be linked or set up in a network with other containers.</li>
<li>The data can be separated in a volume, the directory to be mapped is <code>/ORCL</code>.</li>
<li>Remember to change the password of the <code>sys</code> user (default is <code>Oradoc_db1</code>). This probably should be done in a Dockerfile that
uses this image as base.</li>
<li>There's a smaller image (<code>store/oracle/database-enterprise:12.2.0.1-slim</code>) whose Oracle installation has fewer options and tools.
This is what I'd probably use if I have to work with Oracle again.</li>
</ul>
<p>And that's it for now. If you have any problems or corrections, let me know in the comments!</p>Continuous delivery with Gitlab2016-10-20T00:00:00-05:002023-12-02T02:09:16-05:00Luis Orduztag:luord.com,2016-10-20:/2016/10/20/cd<p>A rapid delivery pipeline using containers and gitlab ci for free (thanks to google app engine). Pretty good for development, I believe.</p><p>When I worked full-time in unique projects, writing the code and tests before running everything manually and then
deploying to staging or production, also manually, was good enough. Now, however, with the potential
to work in several and vastly different projects and environments, this "process" has become increasingly
tedious; as such, as I've become more interested in operations since I started working with Docker, I tasked myself with automating this.</p>
<p>After a while of trying different tools, and the impossibility to work with others due to the limited income of a recent freelancer,
I've settled in a process that I believe will suit me just fine, thanks to GitLab.com's
all-around awesomeness. It beats the other offerings I considered by a margin:</p>
<ul>
<li>Over GitHub, GitLab has private repositories, something clients <em>will</em> want, in its free tier and their built-in CI.</li>
<li>Over Google Cloud Repositories, GitLab has their integrated CI.<sup id="fnref:gcr"><a class="footnote-ref" href="#fn:gcr">1</a></sup></li>
<li>Over Heroku (which I was reticent about anyway for different reasons), GitLab CI is far less opinionated and
offers more freedom in setting up the delivery process while allowing itself to deploy different types of applications more easily.</li>
<li>Over dedicated automation tools such as Jenkins or Buildbot, GitLab CI has the advantage of being simpler and straightforward.
It might not be as maneuverable, but I believe what it offers is more than I need.</li>
</ul>
<p>Finally, GitLab has the advantage that its CI service is fully integrated and out-of-the-box with the git repositories, along with other useful
or potentially useful features. One of these is that it works extremely well with docker, which I already use for local development.</p>
<p>My development process is now, roughly:</p>
<ul>
<li>Running docker containers locally.</li>
<li>Using git hooks (set in place using bash scripts) to trigger tests.</li>
<li>Pushing to GitLab, where their CI will take charge of running tests again and, on success, pushing to the defined destination.</li>
</ul>
<p>Right now, I'm pushing to Google App Engine, whose free tier, despite their lacking repositories<sup id="fnref:rant"><a class="footnote-ref" href="#fn:rant">2</a></sup>, is still the best option for me.</p>
<p>But enough of introduction, let's get on with the guide:</p>
<h2>Preparation</h2>
<p>We need:</p>
<ul>
<li>An account in <a href="https://gitlab.com">GitLab.com</a> or one's own GitLab server. I believe a private GitLab server would work too but I've only tested this
on the website.</li>
<li>An account in <a href="https://cloud.google.com">Google Cloud</a>. Or adjust the <abbr title="Google App Engine">GAE</abbr>-specific steps to your vendor of choice.</li>
<li><a href="https://docker.com">Docker</a>, <a href="https://docs.docker.com/compose/">docker-compose</a>, and <a href="https://git-scm.com">git</a> installed locally. Some familiarity with git might be required.</li>
</ul>
<h2><abbr title="Google App Engine">GAE</abbr> Setup</h2>
<p>(If you're already familiar with creating a project in Google App Engine or use a different vendor, skip ahead to <a href="#gitlab-setup">GitLab setup</a>).</p>
<p>In the cloud console, create a project:</p>
<p><img alt="GAE create project" src="/assets/img/gae/create_project.jpg"></p>
<p>Fill in the name you want (I called mine "gitlab-test").</p>
<p><img alt="GAE project name" src="/assets/img/gae/project_name.jpg"></p>
<p>For all of the following steps, remember the project ID that was returned upon creation.</p>
<p>Afterwards, go to IAM & Admin:</p>
<p><img alt="GAE IAM" src="/assets/img/gae/gae_iam.jpg"></p>
<p>Once there, click on "Service Accounts":</p>
<p><img alt="GAE IAM Services" src="/assets/img/gae/gae_services.jpg"></p>
<p>Then click on "CREATE SERVICE ACCOUNT" and fill in the form that pops up like so:</p>
<p><img alt="GAE Gitlab Service" src="/assets/img/gae/create_service.jpg"></p>
<p><img alt="GAE Gitlab Service" src="/assets/img/gae/gitlab_service.jpg"></p>
<p>After clicking "CREATE", this will download a json file that contains the key that GitLab CI needs to connect to <abbr title="Google App Engine">GAE</abbr>.</p>
<p>Now we need to enable the two APIs required to deploy to app engine remotely. There's a straightforward way that is merely clicking a link
but, since I wouldn't trust it myself if I didn't see it firsthand, I won't expect you to trust it either. So the slightly longer way it is:</p>
<p>In the google cloud console sidebar, click on API manager:</p>
<p><img alt="GAE API Manager" src="/assets/img/gae/api_manager.jpg"></p>
<p>Once in the API manager, click in "ENABLE API".</p>
<p><img alt="GAE Enable API" src="/assets/img/gae/enable_api.jpg"></p>
<p>In the library that opens there's a search box, type "app engine admin api" there and click in the first result.</p>
<p><img alt="GAE API search" src="/assets/img/gae/api_search.jpg"></p>
<p>Once there, click in "ENABLE".</p>
<p><img alt="GAE API Enable" src="/assets/img/gae/api_enable.jpg"></p>
<p>Now just repeat this process (API Manager > Enable API > Search > Enable) for "Google Cloud Storage".</p>
<h2 id="gitlab-setup">GitLab setup</h2>
<p>(If you're already familiar with creating a GitLab project and setting up project variables, go straight to <a href="#code">Code</a>).</p>
<p>First of all, create a project by clicking on the "New Project" button:</p>
<p><img alt="GitLab New Project" src="/assets/img/gitlab/new_project.jpg"></p>
<p>Fill up the "new project" form using the settings and name you want:</p>
<p><img alt="GitLab Create Project" src="/assets/img/gitlab/create_project.jpg"></p>
<p>After sending the form, click on the project settings menu and select "Variables":</p>
<p><img alt="GitLab Variables" src="/assets/img/gitlab/project_settings.jpg"></p>
<p>Set the <code>GAE_PROJECT</code> variable with the id of your Google Cloud Project:</p>
<p><img alt="GitLab Set Variable" src="/assets/img/gitlab/variable.jpg"></p>
<p>Afterwards, set a new variable named <code>GAE_KEY</code>, whose value must be the contents of the json file we downloaded earlier from <abbr title="Google App Engine">GAE</abbr>. Delete the json file as it could be dangerous to have it lying around.</p>
<p>These variables might not be needed if the process to deploy to your vendor doesn't require authentication or there are other ways of authenticating.</p>
<h2 id="code">Code</h2>
<p>The app we're deploying will be a simple "Hello World" in <a href="https://palletsprojects.com/p/flask/">Flask</a> with the following structure:</p>
<div class="highlight"><pre><span></span><code>app
| - app.yaml
| - docker-compose.yml
| - .gitlab-ci.yml
| - app
| | - __init__.py
| | - app.py
| | - test.py
| | - Dockerfile
| | - requirements.txt
</code></pre></div>
<p><code>app.py</code> is within a module and not in the root folder (which would be simpler) for ease of deployment to Google App Engine. This is its code:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">Flask</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">Flask</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span>
<span class="nd">@app</span><span class="o">.</span><span class="n">route</span><span class="p">(</span><span class="s1">'/'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">hello</span><span class="p">():</span>
<span class="k">return</span> <span class="s1">'Hello World!'</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="n">app</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">debug</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</code></pre></div>
<p><code>test.py</code>, as its name indicates, it's just a very simple unit test for <code>app.py</code>, as an example:</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">unittest</span>
<span class="kn">from</span> <span class="nn">app</span> <span class="kn">import</span> <span class="n">app</span>
<span class="k">class</span> <span class="nc">Test</span><span class="p">(</span><span class="n">unittest</span><span class="o">.</span><span class="n">TestCase</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">test</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">app</span><span class="o">.</span><span class="n">test_client</span><span class="p">()</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'/'</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span>
<span class="n">result</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="s1">'utf-8'</span><span class="p">),</span>
<span class="s1">'Hello World!'</span>
<span class="p">)</span>
</code></pre></div>
<p><code>__init__.py</code> has the <abbr title="Google App Engine">GAE</abbr> path setup:</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">os</span><span class="o">,</span> <span class="nn">sys</span>
<span class="n">lib_path</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span>
<span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">abspath</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">dirname</span><span class="p">(</span><span class="vm">__file__</span><span class="p">)),</span>
<span class="s1">'lib'</span>
<span class="p">)</span>
<span class="n">sys</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">insert</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">lib_path</span><span class="p">)</span>
<span class="kn">from</span> <span class="nn">.app</span> <span class="kn">import</span> <span class="n">app</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="n">app</span><span class="o">.</span><span class="n">run</span><span class="p">()</span>
</code></pre></div>
<p>The app only has one dependency, <code>flask</code>, and that single word<sup id="fnref:reqs"><a class="footnote-ref" href="#fn:reqs">3</a></sup> is the content of <code>requirements.txt</code>.</p>
<p>Now the stuff this guide is meant to be about. First the <code>Dockerfile</code>:</p>
<div class="highlight"><pre><span></span><code><span class="k">FROM</span><span class="w"> </span><span class="s">python:latest</span>
<span class="k">ADD</span><span class="w"> </span>requirements.txt<span class="w"> </span>/
<span class="k">RUN</span><span class="w"> </span>pip<span class="w"> </span>install<span class="w"> </span>-r<span class="w"> </span>requirements.txt
<span class="k">ADD</span><span class="w"> </span>.<span class="w"> </span>/code
<span class="k">WORKDIR</span><span class="w"> </span><span class="s">/code</span>
<span class="k">CMD</span><span class="w"> </span><span class="p">[</span><span class="s2">"python"</span><span class="p">,</span><span class="s2">"-m"</span><span class="p">,</span><span class="w"> </span><span class="s2">"unittest"</span><span class="p">,</span><span class="w"> </span><span class="s2">"discover"</span><span class="p">]</span>
</code></pre></div>
<p>Simple enough: from the python image install the requirements and run the test.</p>
<p><code>docker-compose.yml</code> is very simple too:</p>
<div class="highlight"><pre><span></span><code><span class="nt">app</span><span class="p">:</span>
<span class="w"> </span><span class="nt">build</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">app</span>
</code></pre></div>
<p>Build and run what's in the <code>app</code> folder</p>
<p>Now what allows GitLab to perform its magic, the <code>.gitlab-ci.yml</code> file:</p>
<div class="highlight"><pre><span></span><code><span class="nt">back</span><span class="p">:</span>
<span class="w"> </span><span class="nt">image</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">python</span>
<span class="w"> </span><span class="nt">stage</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">build</span>
<span class="w"> </span><span class="nt">script</span><span class="p">:</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="p p-Indicator">>-</span>
<span class="w"> </span><span class="no">pip install -t app/lib</span>
<span class="w"> </span><span class="no">-r app/requirements.txt</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">export PYTHONPATH=$PWD/app/lib:$PYTHONPATH</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">python -m unittest discover</span>
<span class="w"> </span><span class="nt">artifacts</span><span class="p">:</span>
<span class="w"> </span><span class="nt">paths</span><span class="p">:</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">app/lib/</span>
<span class="nt">deploy_production</span><span class="p">:</span>
<span class="w"> </span><span class="nt">image</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">google/cloud-sdk</span>
<span class="w"> </span><span class="nt">stage</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">deploy</span>
<span class="w"> </span><span class="nt">environment</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">production</span>
<span class="w"> </span><span class="nt">script</span><span class="p">:</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">echo $GAE_KEY > /tmp/key.json</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">gcloud config set project $GAE_PROJECT</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="p p-Indicator">>-</span>
<span class="w"> </span><span class="no">gcloud</span>
<span class="w"> </span><span class="no">auth activate-service-account</span>
<span class="w"> </span><span class="no">--key-file /tmp/key.json</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">gcloud --quiet app deploy</span>
<span class="w"> </span><span class="nt">after_script</span><span class="p">:</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">rm /tmp/key.json</span>
</code></pre></div>
<p>There are a couple things happening here, but nothing overly complicated:</p>
<ul>
<li>In the build stage, run the python docker image, install the requirements locally in a folder called <code>lib</code>, run the tests and then make the <code>lib</code> folder available for next stages.</li>
<li>In the deployment stage... deploy the app to <abbr title="Google App Engine">GAE</abbr> (Adjust the commands for your vendor of choice).<sup id="fnref:credit"><a class="footnote-ref" href="#fn:credit">4</a></sup></li>
</ul>
<p>As you can see, to deploy we are using the variables (<code>GAE_KEY</code> and <code>GAE_PROJECT</code>) we set in the previous section.</p>
<p>Finally, <code>app.yaml</code>, which is specific to <abbr title="Google App Engine">GAE</abbr>:</p>
<div class="highlight"><pre><span></span><code><span class="nt">runtime</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">python27</span>
<span class="nt">threadsafe</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">true</span>
<span class="nt">handlers</span><span class="p">:</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">url</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">/</span>
<span class="w"> </span><span class="nt">script</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">app.app</span>
</code></pre></div>
<p>This uses the module structure so it can use the external libraries (<code>flask</code>) in the project.</p>
<h2>Deployment</h2>
<p>There's not much to this, just run <code>docker-compose up</code> and wait for the OK or possible errors. If there's nothing wrong, then we're ready to deploy. The <code>Dockerfile</code> and the <code>docker-compose.yml</code> file can be tweaked to actually run the server or perform any other task one might need.</p>
<p>In the root folder of our app, initialize git and add the repository URL of the gitlab project as remote. Then you only have to push the code
and, after a few minutes, check the url <code>[your-project-id].appspot.com</code> and the "Hello World!" should be staring right back at you.</p>
<h2>Conclusion</h2>
<p>So that's it! We've deployed our app to <abbr title="Google App Engine">GAE</abbr> using GitLab. From then on, you can just dedicate yourself to writing the code and its tests.
To deploy (to production or staging or any environment you choose), you just need to push and this process will take care testing and delivering the code if there aren't any errors.</p>
<p>Things we could do now is setting up automatic local testing on each commit, multiple stages and notifications for failed and successful builds, etc.</p>
<p>If you have any questions, let me know in the comments.</p>
<style>
p > img {
width: 100%;
display: block;
}
</style>
<div class="footnote">
<hr>
<ol>
<li id="fn:gcr">
<p>It amazes me how cloud repositories is almost completely isolated
from all other Google Cloud services. They used to have a Push-to-Deploy feature but that's gone (if it isn't,
it must be really well-hidden now because I spent days reading documentation, forums and question threads about this) and now
they suggest setting up one's own continuous integration service. I can't imagine why they did that and, again,
I'd rather not risk being charged for running their recommended Jenkins setup. <a class="footnote-backref" href="#fnref:gcr" title="Jump back to footnote 1 in the text">↩</a></p>
</li>
<li id="fn:rant">
<p>Indeed, if their cloud repositories were integrated with their cloud platform, I might have never bothered to look into GitLab. A good thing in hindsight, all things considered. <a class="footnote-backref" href="#fnref:rant" title="Jump back to footnote 2 in the text">↩</a></p>
</li>
<li id="fn:reqs">
<p>Not versioning your dependencies is, of course, not recommended. <a class="footnote-backref" href="#fnref:reqs" title="Jump back to footnote 3 in the text">↩</a></p>
</li>
<li id="fn:credit">
<p>The script for the <code>deploy_production</code> stage in <code>.gitlab-ci.yml</code> is partly based on the one in <a href="https://medium.com/evenbit/an-easy-guide-to-automatically-deploy-your-google-app-engine-project-with-gitlab-ci-48cb84757125">this</a> cool post by Dennis Alund. <a class="footnote-backref" href="#fnref:credit" title="Jump back to footnote 4 in the text">↩</a></p>
</li>
</ol>
</div>The Not-Invented-Here syndrome2016-06-25T00:00:00-05:002023-12-02T02:09:16-05:00Luis Orduztag:luord.com,2016-06-25:/2016/06/25/nih<p>On creating own solutions versus implementing existing libraries or methods.</p><p>I read a while ago about the NIH syndrome and how it's generally not recommended because it unnecessarily increases the workload and the amount of code that needs to be maintained. Not to mention that using existing libraries or frameworks, specially open source ones, can also eventually involve helping the community and, thus, improving the code for everyone.</p>
<p>So, the recommendation is generally reusing as much code as possible, hopefully keeping the amount of original code reduced to the actual business logic of the project at hand. I think this is sound, but there's something to be said about producing in-house code and reinventing the wheel a little.</p>
<p>In my last official project, where I worked as a backend developer for a startup, we did use frameworks for all of backend, frontend and presentation, as well as several plugins for the frameworks to avoid increasing the workload too much, but we also wrote a lot of code that we might have found in existing libraries if we looked.</p>
<p>For a lot of the REST API, for example, I wrote all of the entrypoints and callback logic. I know now that proper usage of <a href="https://flask-restful.readthedocs.io/" title="One of the best plugins for Flask">Flask-Restful</a> (<a href="https://palletsprojects.com/p/flask/" title="Flask Framework">flask</a> is, of course, my favorite framework) could have saved me a lot of work in that area... But I don't really regret it, I can say a learned a whole lot because I've always been a bit of a hands-on learner.</p>
<p>Of course, I've also seen first-hand that doing everything in-house can and does get out of control and, after a while, it becomes almost impossible for the handful of developers of a startup to maintain all that code.</p>
<p>In the project I'm working now, thanks to what I learned from writing a lot of my own code, it's been easier for me to research libraries and decide what would be a better fit as well as recognizing where I really do need to write; part of the reason I didn't reuse as much as I could in that other project was overestimating what was actual business logic and what were mere building blocks.</p>
<p>Given the chance, I'll probably refactor all that code and use Flask-Restful or similar to simplify it and make it more easily maintainable; but every learning opportunity is a good opportunity so I'm glad I went more zealous in the first go in that project.</p>
<p>What I'm trying to say is yes, one should avoid the NIH syndrome, surf the community, reuse stuff that hundreds if not thousands of people have polished (the more eyes, the better) and prevent getting the codebase from getting out of control due to reinventing the wheel. But one should also tackle at least one project where one writes as much code as possible, it highlights the importance of reusing the code in later projects, one gets first-hand experience on what leads people to write such libraries in the first place and, in general, one learns how the kind of projects one is working on generally work.</p>
<p>It goes without saying, of course, that doing such a thing is really only beneficial early in one's career. I see no reason for reinventing the wheel once one is already an experimented developer. Of course, someone more experienced than me could probably tell me otherwise.</p>
<p>In a different but related matter, in this website I've tried to avoid using frameworks of any kind; opting for more hands-on code. Just like doing it once helps to learn, I think that keeping a side, personal, project for practice keeps one from forgetting the basics. This site is, thus, my sandbox in a way, helping me practice HTML and Jinja templating (through the Pelican blog), and LESS and CSS for the themes.</p>Hello World!2016-03-14T00:00:00-05:002023-12-02T02:09:16-05:00Luis Orduztag:luord.com,2016-03-14:/2016/03/14/hello<p>I, Luis Orduz, hereby start my blog.</p><p>I'm a Software Developer and, as such, the first entry of my blog is
to be a <em>Hello World</em> page, which I now proceed to write in Python...</p>
<div class="highlight"><pre><span></span><code><span class="nb">print</span><span class="p">(</span><span class="s2">"Ahoy!"</span><span class="p">)</span>
</code></pre></div>
<p>... and browser ECMAScript (or JavaScript if you prefer, which I don't).</p>
<div class="highlight"><pre><span></span><code><span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">"Good Day."</span><span class="p">)</span>
</code></pre></div>
<p>So... that's it for today.</p>
<p>UPDATE: I decided to dedicate my site mostly to Software after all.</p>