Jekyll2023-10-12T19:51:22+00:00http://alanparr.github.io/feed.xmlUse Small IconsThis is my very occasionally updated blog.Alan ParrIncremental Refactoring for the Win2023-10-12T00:00:00+00:002023-10-12T00:00:00+00:00http://alanparr.github.io/incremental-refactoring-for-the-win<p>We’ve all encountered those sprawling, unruly code bases that have grown unchecked over the years. We label them as “legacy,” but that is often just code for “code that works, pays the bills, and our customers rely on to continue working”. The challenge then becomes how to improve these code bases without causing more harm than good.</p>
<p>In this article, we’ll explore a pragmatic approach to rejuvenating legacy code through small, incremental changes. This method is all about making minimal, yet meaningful, alterations and getting them into production quickly. While it might appear counterintuitive at first, this strategy can save you time and frustration in the long run. So, let’s dive into the art of incremental refactoring.</p>
<ol>
<li>
<p>Redefining “Legacy” Code:
Before we dive into the how, let’s reconsider what we mean by “legacy” code. It’s not just outdated, messy code; it’s the code that pays the bills and is relied upon by our users.</p>
</li>
<li>
<p>The Power of Incremental Changes:
Small, incremental improvements are the cornerstone of this approach. Whether it’s refactoring a single method, modernizing a class, or cleaning up a snippet of unused code, the goal is to make the smallest possible change and push it to production as quickly as possible. While this may seem like a time-consuming process due to code reviews, QA testing, merging, and publishing, it ultimately saves time by preventing issues that large, sweeping changes can introduce.</p>
</li>
<li>
<p>The Importance of Testing:
In the world of legacy code, poor code coverage is often the norm. Testing may seem daunting, but it’s crucial to ensure the reliability of your changes. Start by writing tests for the specific code you’re altering, and gradually expand your test suite as you make further improvements. A test that just calls your method and verifies it doesn’t throw an exception is better than nothing.</p>
</li>
<li>
<p>Embrace Imperfection:
Your initial refactoring efforts don’t have to result in a gold-standard SOLID-adherent perfect vision of code. Refactoring from “terrible” to “bad” is still an improvement. Subsequent iterations can then raise the code’s quality from “bad” to “okay” and beyond.</p>
</li>
<li>
<p>The Pitfalls of Radical Overhauls:
Attempting a complete codebase overhaul in one go can be a recipe for disaster. It’s highly likely to introduce more bugs and leave you questioning your decisions. Incremental changes, on the other hand, allow you to maintain confidence in your work and monitor the impact of each alteration.</p>
</li>
<li>
<p>Tailoring Change Sizes to Test Coverage:
If your codebase lacks substantial test coverage, stick to small-scale changes to minimize risks. As your testing improves, you can gradually expand the scope of your refactorings.</p>
</li>
</ol>
<p>Let’s see this process in action with a simple example:</p>
<h2 id="starting-point">Starting Point</h2>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">class</span> <span class="nc">Bad</span>
<span class="p">{</span>
<span class="k">private</span> <span class="n">DatabaseService</span> <span class="n">_database</span><span class="p">;</span>
<span class="k">public</span> <span class="nf">Bad</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">_database</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">DatabaseService</span><span class="p">(</span><span class="n">System</span><span class="p">.</span><span class="n">Configuration</span><span class="p">.</span><span class="n">ConfigurationManager</span><span class="p">.</span><span class="n">AppSettings</span><span class="p">[</span><span class="s">"DatabaseConnectionString"</span><span class="p">]);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This class is problematic for a few reasons</p>
<ul>
<li>It news up the DatabaseService which means we can’t replace it during testing (“new is glue”).</li>
<li>It is coupled to the concrete type which is also a barrier to testing.</li>
<li>It reaches out in to app configuration to get the connection string for that database. This restricts our options for changing where that configuration comes from and also gives the caller too much knowledge about the internal workings of that class and makes it difficult to make changes to how the class is instantiated.</li>
</ul>
<h2 id="better">Better</h2>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">class</span> <span class="nc">Better</span>
<span class="p">{</span>
<span class="k">private</span> <span class="n">DatabaseService</span> <span class="n">_database</span><span class="p">;</span>
<span class="k">public</span> <span class="nf">Better</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">_database</span> <span class="p">=</span> <span class="n">DatabaseServiceFactory</span><span class="p">.</span><span class="nf">GetDatabase</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This is better as have isolated the logic of building a DatabaseService instance in to a single place. It should also be fairly easy to drop this in the place of the previous code with no other changes.</p>
<p>But it still has some issues:</p>
<ul>
<li>We are now bound to a static factory but at least now we no longer know anything about the object that is being created.</li>
<li>We are still bound to the concrete type.</li>
</ul>
<h2 id="better-still">Better Still</h2>
<p>Ideally you would skip this step and go straight to the Final Form but there are many reasons why this may not be immediately possible, for instance you don’t have DI currently, so I wanted to show this step as an example of how to achieve testability when you do not have the benefits of DI.</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">class</span> <span class="nc">BetterStill</span>
<span class="p">{</span>
<span class="k">private</span> <span class="n">IDatabaseService</span> <span class="n">database</span><span class="p">;</span>
<span class="k">public</span> <span class="nf">BetterStill</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">database</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">DatabaseServiceFactory</span><span class="p">().</span><span class="nf">GetDatabase</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">class</span> <span class="nc">DatabaseServiceFactory</span>
<span class="p">{</span>
<span class="k">private</span> <span class="k">static</span> <span class="n">IDatabaseService</span> <span class="n">_testInstance</span><span class="p">;</span>
<span class="k">public</span> <span class="n">IDatabaseService</span> <span class="nf">GetDatabase</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">if</span><span class="p">(</span><span class="n">_testInstance</span> <span class="p">!=</span> <span class="k">null</span><span class="p">)</span>
<span class="k">return</span> <span class="n">_testInstance</span><span class="p">;</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">DatabaseService</span><span class="p">(</span><span class="n">System</span><span class="p">.</span><span class="n">Configuration</span><span class="p">.</span><span class="n">ConfigurationManager</span><span class="p">.</span><span class="n">AppSettings</span><span class="p">[</span><span class="s">"DatabaseConnectionString"</span><span class="p">]);</span>
<span class="p">}</span>
<span class="k">protected</span> <span class="k">void</span> <span class="nf">SetTestInstance</span><span class="p">(</span><span class="n">IDatabaseService</span> <span class="n">instance</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">_testInstance</span> <span class="p">=</span> <span class="n">instance</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Now we have an interface. The combination of this and modifying the DatabaseServiceFactory now unlocks substitution of the database service.</p>
<p>Now before anyone gets upset, this is still not good code, it is just a step closer to where we want to be. We have actually introduced a bit of sin in the form of having affordances for testing in a production class (the _testInstance logic in DatabaseServiceFactory) but it is for a good cause. We can now create a TestDatabaseServiceFactory in our tests that will allow us to override the IDatabaseService instance in testing.</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">class</span> <span class="nc">TestDatabaseServiceFactory</span> <span class="p">:</span> <span class="n">DatabaseServiceFactory</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">SetInstance</span><span class="p">(</span><span class="n">IDatabaseService</span> <span class="n">instance</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">base</span><span class="p">.</span><span class="nf">SetTestInstance</span><span class="p">(</span><span class="n">instance</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>which we can use as</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="n">SetUp</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">TestSetup</span><span class="p">()</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">databaseInstance</span> <span class="p">=</span> <span class="k">new</span> <span class="n">Mock</span><span class="p"><</span><span class="n">IDatabaseService</span><span class="p">>();</span>
<span class="n">TestDatabaseServiceFactory</span><span class="p">.</span><span class="nf">SetInstance</span><span class="p">(</span><span class="n">databaseInstance</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>
<p>So now we can inject a Mock and start mocking and verifying behaviour of this service.</p>
<p>The reason for the TestDatabaseServiceFactory type is to allow us to keep the SetTestInstance method on the base class protected and prevent (or at least reduce the chances) of someone accidentally using it in Production.</p>
<p>But don’t stop here.</p>
<h2 id="final-form">Final form</h2>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">class</span> <span class="nc">Final</span>
<span class="p">{</span>
<span class="k">private</span> <span class="n">IDatabaseService</span> <span class="n">_database</span><span class="p">;</span>
<span class="k">public</span> <span class="nf">Final</span><span class="p">(</span><span class="n">IDatabaseService</span> <span class="n">database</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">_database</span> <span class="p">=</span> <span class="n">database</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This is the final form for this example.</p>
<ul>
<li>We are no longer newing up the instance directly with it now coming in on the constructor and it is now an interface so we are no longer tied to a concrete type, these 2 things together has unlocked testability in our class.</li>
<li>In production, if we need to change how the DatabaseService is created we can now do that easily without having to make any changes to its consumers.</li>
</ul>
<h2 id="conclusion">Conclusion</h2>
<p>Legacy codebases are not monuments to be razed and rebuilt in a day. They are the foundation of your projects and require a thoughtful, incremental approach to improve them. By making small, well-tested changes and embracing gradual progress, you can bring your legacy code back to life, ensuring its reliability and maintainability while avoiding the pitfalls of radical overhauls.</p>Alan ParrWe’ve all encountered those sprawling, unruly code bases that have grown unchecked over the years. We label them as “legacy,” but that is often just code for “code that works, pays the bills, and our customers rely on to continue working”. The challenge then becomes how to improve these code bases without causing more harm than good.Tips for building effective teams2023-09-15T00:00:00+00:002023-09-15T00:00:00+00:00http://alanparr.github.io/tips-for-effective-teams<p>Tips for building effective teams</p>
<h2 id="1-trust-your-team">1 Trust your team</h2>
<p>You do not know everything. Your team may want to use a new technology or approach to solve a problem, give them the freedom to try new things but be prepared to call time on it and steer them back on to a good path if it isn’t working out.</p>
<h2 id="2-ensure-your-team-can-trust-you">2 Ensure your team can trust you</h2>
<p>Do not promise anything to your team unless you a) intend to follow through on it and b) know you have the power/autonomy within the business to follow through on it without being overridden by someone higher up.</p>
<p>You can keep your promise 10 times but you’ll only get away with breaking it once, maybe twice before you lose the trust of your team and have to start from zero.</p>
<p>Even if they know you’ve been overridden by someone higher up, if this happens enough times they’ll start to see your promises as empty and their fulfilment ultimately out of your control.</p>
<h2 id="3-be-transparent">3 Be transparent</h2>
<p>Or as transparent as you can be within the business you work in. Let them know what challenges you are having, let them know what jobs are coming up and what may have to wait for a while.</p>
<p>If they’ve asked for something, don’t let it drop. If there is a delay, tell them otherwise they will assume you’ve just forgotten/don’t care about it and this will make #2 more difficult.</p>
<p>They can’t understand the business without a complete picture.</p>
<p>The more they know the more effective they will be.</p>
<h2 id="4-create-a-safe-space-for-experimentation">4 Create a safe space for experimentation</h2>
<p>Tangentially linked to #1, allow your team to experiment but make sure they know that their experiments can not achieve their expected outcome without fear or reprisal or loss of face.</p>
<p>Note how I didn’t use the word failure. Even if an experiment doesn’t achieve the expected outcome you will always come out of it with more information than you started with which is in itself a positive outcome and will increase the chance of your next experiment being successful.</p>
<h2 id="5-its-okay-to-take-a-couple-of-attempts-to-solve-a-problem">5 It’s okay to take a couple of attempts to solve a problem</h2>
<p>I’ve found this to be especially true when working in large, poorly-architected legacy codebases where there is always <em>that service</em> that does far too much and everyone is really scared of touching it. When they’re tackling a big problem in this kind of situation, make clear that you know it may take a couple of attempts to get to the complete solution and that is okay and not succeeding first time isn’t a sign of failure.</p>
<p>This will make people a lot less scared of making changes which will empower them to make difficult improvements that they otherwise may avoid doing.</p>
<h2 id="6-mistakes-are-part-of-life">6 Mistakes are part of life</h2>
<p>In the words of the great Jean-Luc Picard</p>
<p><img src="\assets\img\picard-mistakes.png" alt="It is possible to commit no mistakes and still lose. That is not weakness, that is life." /></p>
<p>We can’t know everything. Every mistake is a lesson learned. Every <em>“oh it does that too?”</em> moment is knowledge gained.</p>Alan ParrTips for building effective teamsMeasure everything - Empty collections2022-02-13T00:00:00+00:002022-02-13T00:00:00+00:00http://alanparr.github.io/measure-everything-empty-collections<p>What’s the quickest way to get an empty <code class="language-plaintext highlighter-rouge">IEnumerable<string></code>, <code class="language-plaintext highlighter-rouge">List<string></code>, or <code class="language-plaintext highlighter-rouge">string[]</code>? Let’s measure it.</p>
<h2 id="the-test">The Test</h2>
<p>As usual, code is running in a <a href="https://www.linqpad.net/">Linqpad</a> script and is using <a href="https://github.com/dotnet/BenchmarkDotNet">Benchmark.Net</a>.</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">#</span><span class="n">LINQPad</span> <span class="n">optimize</span><span class="p">+</span> <span class="c1">// Enable compiler optimizations</span>
<span class="k">void</span> <span class="nf">Main</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">Util</span><span class="p">.</span><span class="n">AutoScrollResults</span> <span class="p">=</span> <span class="k">true</span><span class="p">;</span>
<span class="n">BenchmarkRunner</span><span class="p">.</span><span class="n">Run</span><span class="p"><</span><span class="n">EmptyCollectionsBenchmark</span><span class="p">>();</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">ShortRunJob</span><span class="p">,</span> <span class="n">MemoryDiagnoserAttribute</span><span class="p">]</span>
<span class="k">public</span> <span class="k">class</span> <span class="nc">EmptyCollectionsBenchmark</span>
<span class="p">{</span>
<span class="p">[</span><span class="n">Benchmark</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">Enumerable_EnumerableEmpty</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">IEnumerable</span><span class="p"><</span><span class="kt">string</span><span class="p">></span> <span class="n">val</span> <span class="p">=</span> <span class="n">Enumerable</span><span class="p">.</span><span class="n">Empty</span><span class="p"><</span><span class="kt">string</span><span class="p">>();</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Benchmark</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">Enumerable_ArrayEmpty</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">IEnumerable</span><span class="p"><</span><span class="kt">string</span><span class="p">></span> <span class="n">val</span> <span class="p">=</span> <span class="n">Array</span><span class="p">.</span><span class="n">Empty</span><span class="p"><</span><span class="kt">string</span><span class="p">>();</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Benchmark</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">List_EnumerableEmpty</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">List</span><span class="p"><</span><span class="kt">string</span><span class="p">></span> <span class="n">val</span> <span class="p">=</span> <span class="n">Enumerable</span><span class="p">.</span><span class="n">Empty</span><span class="p"><</span><span class="kt">string</span><span class="p">>().</span><span class="nf">ToList</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Benchmark</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">List_ArrayEmpty</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">List</span><span class="p"><</span><span class="kt">string</span><span class="p">></span> <span class="n">val</span> <span class="p">=</span> <span class="n">Array</span><span class="p">.</span><span class="n">Empty</span><span class="p"><</span><span class="kt">string</span><span class="p">>().</span><span class="nf">ToList</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Benchmark</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">List_NewList</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">List</span><span class="p"><</span><span class="kt">string</span><span class="p">></span> <span class="n">val</span> <span class="p">=</span> <span class="k">new</span> <span class="n">List</span><span class="p"><</span><span class="kt">string</span><span class="p">>();</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Benchmark</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">Array_ArrayEmpty</span><span class="p">()</span>
<span class="p">{</span>
<span class="kt">string</span><span class="p">[]</span> <span class="n">val</span> <span class="p">=</span> <span class="n">Array</span><span class="p">.</span><span class="n">Empty</span><span class="p"><</span><span class="kt">string</span><span class="p">>();</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Benchmark</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">Array_NewArray_ZeroLength</span><span class="p">()</span>
<span class="p">{</span>
<span class="kt">string</span><span class="p">[]</span> <span class="n">val</span> <span class="p">=</span> <span class="k">new</span> <span class="kt">string</span><span class="p">[</span><span class="m">0</span><span class="p">];</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Benchmark</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">Array_NewArray_100Length</span><span class="p">()</span>
<span class="p">{</span>
<span class="kt">string</span><span class="p">[]</span> <span class="n">val</span> <span class="p">=</span> <span class="k">new</span> <span class="kt">string</span><span class="p">[</span><span class="m">100</span><span class="p">];</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<h2 id="the-results">The Results</h2>
<table>
<thead>
<tr>
<th>Method</th>
<th style="text-align: right">Mean</th>
<th style="text-align: right">Error</th>
<th style="text-align: right">StdDev</th>
<th style="text-align: right">Median</th>
<th style="text-align: right">Gen 0</th>
<th style="text-align: right">Allocated</th>
</tr>
</thead>
<tbody>
<tr>
<td>Enumerable_EnumerableEmpty</td>
<td style="text-align: right">0.0262 ns</td>
<td style="text-align: right">0.8292 ns</td>
<td style="text-align: right">0.0455 ns</td>
<td style="text-align: right">0.0000 ns</td>
<td style="text-align: right">-</td>
<td style="text-align: right">-</td>
</tr>
<tr>
<td>Enumerable_ArrayEmpty</td>
<td style="text-align: right">0.0312 ns</td>
<td style="text-align: right">0.9871 ns</td>
<td style="text-align: right">0.0541 ns</td>
<td style="text-align: right">0.0000 ns</td>
<td style="text-align: right">-</td>
<td style="text-align: right">-</td>
</tr>
<tr>
<td>List_EnumerableEmpty</td>
<td style="text-align: right">16.3021 ns</td>
<td style="text-align: right">18.8314 ns</td>
<td style="text-align: right">1.0322 ns</td>
<td style="text-align: right">15.7586 ns</td>
<td style="text-align: right">0.0076</td>
<td style="text-align: right">32 B</td>
</tr>
<tr>
<td>List_ArrayEmpty</td>
<td style="text-align: right">25.2154 ns</td>
<td style="text-align: right">24.0245 ns</td>
<td style="text-align: right">1.3169 ns</td>
<td style="text-align: right">24.7718 ns</td>
<td style="text-align: right">0.0076</td>
<td style="text-align: right">32 B</td>
</tr>
<tr>
<td>List_NewList</td>
<td style="text-align: right">5.2154 ns</td>
<td style="text-align: right">3.8314 ns</td>
<td style="text-align: right">0.2100 ns</td>
<td style="text-align: right">5.1523 ns</td>
<td style="text-align: right">0.0076</td>
<td style="text-align: right">32 B</td>
</tr>
<tr>
<td>Array_ArrayEmpty</td>
<td style="text-align: right">0.0035 ns</td>
<td style="text-align: right">0.1098 ns</td>
<td style="text-align: right">0.0060 ns</td>
<td style="text-align: right">0.0000 ns</td>
<td style="text-align: right">-</td>
<td style="text-align: right">-</td>
</tr>
<tr>
<td>Array_NewArray_ZeroLength</td>
<td style="text-align: right">3.0829 ns</td>
<td style="text-align: right">1.0481 ns</td>
<td style="text-align: right">0.0574 ns</td>
<td style="text-align: right">3.0888 ns</td>
<td style="text-align: right">0.0057</td>
<td style="text-align: right">24 B</td>
</tr>
<tr>
<td>Array_NewArray_100Length</td>
<td style="text-align: right">42.2577 ns</td>
<td style="text-align: right">23.3965 ns</td>
<td style="text-align: right">1.2824 ns</td>
<td style="text-align: right">41.6450 ns</td>
<td style="text-align: right">0.1970</td>
<td style="text-align: right">824 B</td>
</tr>
</tbody>
</table>
<h2 id="the-conclusion">The Conclusion</h2>
<p>So what is the most efficient way to get an empty collection?</p>
<ul>
<li>For an IEnumerable, <code class="language-plaintext highlighter-rouge">Enumerable.Empty<T>()</code>, which is what I was expecting.</li>
<li>For a List, <code class="language-plaintext highlighter-rouge">new List<T>()</code> is the winner. I initially expected one of the <code class="language-plaintext highlighter-rouge">.Empty()</code> methods to win but now that I think about it, it makes sense that they don’t given the <code class="language-plaintext highlighter-rouge">ToList()</code> call is probably initiating an enumeration or boxing operation.</li>
<li>For an array, <code class="language-plaintext highlighter-rouge">Array.Empty<T>()</code> is the winner. Again, no surprise here.</li>
</ul>
<p>I hope that was informative. If there are any flaws in the above that may be rendering the results inaccurate, please leave a comment.</p>Alan ParrWhat’s the quickest way to get an empty IEnumerable<string>, List<string>, or string[]? Let’s measure it.Customising JSON serialization with Json.Net2022-02-04T00:00:00+00:002022-02-04T00:00:00+00:00http://alanparr.github.io/customising-json-serialization<p>While integrating with the API of a popular tropical-sounding online shop, I hit a problem where their Sandbox API was rejecting a perfectly valid request that the main API would most likely be fine with.</p>
<p>In this case, it was objecting to decimals being rendered as <code class="language-plaintext highlighter-rouge">10.0</code> instead of <code class="language-plaintext highlighter-rouge">10</code>, likely because the sandbox is auto-generated from the Swagger models and was just doing a string comparison on the request and was expecting <code class="language-plaintext highlighter-rouge">10</code>.
As soon as we removed the trailing <code class="language-plaintext highlighter-rouge">.0</code>, the request was accepted.</p>
<p>Rather than just accept the sandbox was broken, I decided to see if it was possible to fix this, so I looked in to Converters so I could intercept serialization of the types and make my tweaks and this is what I came up with:</p>
<p>As per usual, this is in the form of a Linqpad script, so just add the JSON.Net Nuget package and drop in the below code.</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">void</span> <span class="nf">Main</span><span class="p">()</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">x</span> <span class="p">=</span> <span class="k">new</span> <span class="n">Foo</span> <span class="p">{</span><span class="n">Bar</span> <span class="p">=</span> <span class="m">10.0</span><span class="p">};</span>
<span class="n">Newtonsoft</span><span class="p">.</span><span class="n">Json</span><span class="p">.</span><span class="n">JsonConvert</span><span class="p">.</span><span class="nf">SerializeObject</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">Newtonsoft</span><span class="p">.</span><span class="n">Json</span><span class="p">.</span><span class="n">Formatting</span><span class="p">.</span><span class="n">Indented</span><span class="p">).</span><span class="nf">Dump</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">class</span> <span class="nc">Foo</span><span class="p">{</span>
<span class="p">[</span><span class="nf">JsonConverter</span><span class="p">(</span><span class="k">typeof</span><span class="p">(</span><span class="n">DoubleConverter</span><span class="p">))]</span>
<span class="k">public</span> <span class="kt">double</span> <span class="n">Bar</span> <span class="p">{</span><span class="k">get</span><span class="p">;</span><span class="k">set</span><span class="p">;}</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">class</span> <span class="nc">DoubleConverter</span> <span class="p">:</span> <span class="n">JsonConverter</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">override</span> <span class="kt">bool</span> <span class="nf">CanConvert</span><span class="p">(</span><span class="n">Type</span> <span class="n">objectType</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">objectType</span> <span class="p">==</span> <span class="k">typeof</span><span class="p">(</span><span class="kt">double</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">override</span> <span class="kt">object</span> <span class="nf">ReadJson</span><span class="p">(</span><span class="n">JsonReader</span> <span class="n">reader</span><span class="p">,</span> <span class="n">Type</span> <span class="n">objectType</span><span class="p">,</span> <span class="kt">object</span> <span class="n">existingValue</span><span class="p">,</span> <span class="n">JsonSerializer</span> <span class="n">serializer</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="kt">double</span><span class="p">.</span><span class="nf">Parse</span><span class="p">(</span><span class="n">existingValue</span><span class="p">.</span><span class="nf">ToString</span><span class="p">());</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">override</span> <span class="k">void</span> <span class="nf">WriteJson</span><span class="p">(</span><span class="n">JsonWriter</span> <span class="n">writer</span><span class="p">,</span> <span class="kt">object</span> <span class="k">value</span><span class="p">,</span> <span class="n">JsonSerializer</span> <span class="n">serializer</span><span class="p">)</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">asDouble</span> <span class="p">=</span> <span class="p">(</span><span class="kt">double</span><span class="p">)</span><span class="k">value</span><span class="p">;</span>
<span class="kt">var</span> <span class="n">asInt</span> <span class="p">=</span> <span class="p">(</span><span class="kt">int</span><span class="p">)</span><span class="n">asDouble</span><span class="p">;</span>
<span class="kt">var</span> <span class="n">hasFraction</span> <span class="p">=</span> <span class="n">asDouble</span> <span class="p">-</span> <span class="n">asInt</span><span class="p">;</span>
<span class="k">if</span><span class="p">(</span><span class="n">hasFraction</span> <span class="p">></span> <span class="m">0</span><span class="p">){</span>
<span class="n">writer</span><span class="p">.</span><span class="nf">WriteValue</span><span class="p">(</span><span class="n">asDouble</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">else</span><span class="p">{</span>
<span class="n">writer</span><span class="p">.</span><span class="nf">WriteValue</span><span class="p">(</span><span class="n">asInt</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The converter is quite naive, I haven’t yet done the work to fully exercise it to make sure it handles all scenarios.</p>
<p><code class="language-plaintext highlighter-rouge">CanConvert</code> is to tell Json.Net if it can convert the incoming type, which will be true if the type is a double.</p>
<p><code class="language-plaintext highlighter-rouge">ReadJson</code> is used during deserialization and simply parses the double, working on the assumption that it is valid. I’ll likely improve this bit as I use it.</p>
<p><code class="language-plaintext highlighter-rouge">WriteJson</code> is used during serialization, this is the bit I was really interested in. To determine if there is a zero fraction, I get the value as a double, then an int, minus the int from the double and see if I have anything left. If I do, write the double version, else write the int version.</p>
<p>Json.Net knows to use this converter from the attribute placed on the double property</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="nf">JsonConverter</span><span class="p">(</span><span class="k">typeof</span><span class="p">(</span><span class="n">DoubleConverter</span><span class="p">))]</span>
<span class="k">public</span> <span class="kt">double</span> <span class="n">Bar</span> <span class="p">{</span><span class="k">get</span><span class="p">;</span><span class="k">set</span><span class="p">;}</span>
</code></pre></div></div>
<p>Without the attribute, the following Json is rendered</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"Bar"</span><span class="p">:</span><span class="w"> </span><span class="mf">10.0</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>With the attribute, we now get the following Json</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"Bar"</span><span class="p">:</span><span class="w"> </span><span class="mi">10</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>which should make my request succeed.</p>Alan ParrWhile integrating with the API of a popular tropical-sounding online shop, I hit a problem where their Sandbox API was rejecting a perfectly valid request that the main API would most likely be fine with.Measure everything - Count() vs Any()2022-01-28T00:00:00+00:002022-01-28T00:00:00+00:00http://alanparr.github.io/measure-everything-any-vs-count<p>Within any community, there are often a series of untested truths. Within development communities, this often takes the form of X is faster than Y or Z is more efficient than W. While these are often true at some point, we don’t often test them to see if they remain true years later.</p>
<p>One such instance is Count() vs Any() on IEnumerables and Collections. The general wisdom as I understand it is that Any() will generally be faster on IEnumerables and Count is generally better for Collections.</p>
<p>Using the power of Linqpad and Benchmark.Net, let’s see if that is true.</p>
<p>This is the Linqpad script I used, you’ll just need to create a new query, add a reference to Benchmark.Net, and paste this code in:</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">#</span><span class="n">LINQPad</span> <span class="n">optimize</span><span class="p">+</span> <span class="c1">// Enable compiler optimizations</span>
<span class="k">void</span> <span class="nf">Main</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">Util</span><span class="p">.</span><span class="n">AutoScrollResults</span> <span class="p">=</span> <span class="k">true</span><span class="p">;</span>
<span class="n">BenchmarkRunner</span><span class="p">.</span><span class="n">Run</span><span class="p"><</span><span class="n">CountVsAny</span><span class="p">>();</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">ShortRunJob</span><span class="p">,</span> <span class="n">MemoryDiagnoser</span><span class="p">]</span>
<span class="k">public</span> <span class="k">class</span> <span class="nc">CountVsAny</span><span class="p">{</span>
<span class="k">private</span> <span class="n">IEnumerable</span><span class="p"><</span><span class="kt">int</span><span class="p">></span> <span class="nf">GetYieldingEnumerable</span><span class="p">(){</span>
<span class="k">for</span> <span class="p">(</span><span class="kt">int</span> <span class="n">i</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">i</span> <span class="p"><</span> <span class="m">10</span><span class="p">;</span> <span class="n">i</span><span class="p">++)</span>
<span class="p">{</span>
<span class="k">yield</span> <span class="k">return</span> <span class="n">i</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">private</span> <span class="n">IEnumerable</span><span class="p"><</span><span class="kt">int</span><span class="p">></span> <span class="nf">NonYieldingEnumerable</span><span class="p">()</span> <span class="p">=></span> <span class="n">Enumerable</span><span class="p">.</span><span class="nf">Range</span><span class="p">(</span><span class="m">1</span><span class="p">,</span><span class="m">10</span><span class="p">);</span>
<span class="k">private</span> <span class="n">List</span><span class="p"><</span><span class="kt">int</span><span class="p">></span> <span class="nf">GetList</span><span class="p">()</span> <span class="p">=></span> <span class="n">Enumerable</span><span class="p">.</span><span class="nf">Range</span><span class="p">(</span><span class="m">1</span><span class="p">,</span><span class="m">10</span><span class="p">).</span><span class="nf">ToList</span><span class="p">();</span>
<span class="p">[</span><span class="n">Benchmark</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">YieldingEnumerableCountMethod</span><span class="p">(){</span>
<span class="kt">var</span> <span class="n">result</span> <span class="p">=</span> <span class="nf">GetYieldingEnumerable</span><span class="p">().</span><span class="nf">Count</span><span class="p">()</span> <span class="p">></span> <span class="m">0</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Benchmark</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">YieldingEnumerableAnyMethod</span><span class="p">()</span> <span class="p">{</span>
<span class="kt">var</span> <span class="n">result</span> <span class="p">=</span> <span class="nf">GetYieldingEnumerable</span><span class="p">().</span><span class="nf">Any</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Benchmark</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">NonYieldingEnumerableCountMethod</span><span class="p">()</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">result</span> <span class="p">=</span> <span class="nf">NonYieldingEnumerable</span><span class="p">().</span><span class="nf">Count</span><span class="p">()</span> <span class="p">></span> <span class="m">0</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Benchmark</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">NonYieldingEnumerableAnyMethod</span><span class="p">()</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">result</span> <span class="p">=</span> <span class="nf">NonYieldingEnumerable</span><span class="p">().</span><span class="nf">Any</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Benchmark</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">ListCountMethod</span><span class="p">()</span> <span class="p">{</span>
<span class="kt">var</span> <span class="n">result</span> <span class="p">=</span> <span class="nf">GetList</span><span class="p">().</span><span class="nf">Count</span><span class="p">()</span> <span class="p">></span> <span class="m">0</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Benchmark</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">ListCountProperty</span><span class="p">()</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">result</span> <span class="p">=</span> <span class="nf">GetList</span><span class="p">().</span><span class="n">Count</span> <span class="p">></span> <span class="m">0</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Benchmark</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">ListAnyMethod</span><span class="p">()</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">result</span> <span class="p">=</span> <span class="nf">GetList</span><span class="p">().</span><span class="nf">Any</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Running this produced the following results:</p>
<table>
<thead>
<tr>
<th>Method</th>
<th style="text-align: right">Mean</th>
<th style="text-align: right">Error</th>
<th style="text-align: right">StdDev</th>
<th style="text-align: right">Gen 0</th>
<th style="text-align: right">Allocated</th>
</tr>
</thead>
<tbody>
<tr>
<td>YieldingEnumerableCountMethod</td>
<td style="text-align: right">86.50 ns</td>
<td style="text-align: right">140.361 ns</td>
<td style="text-align: right">7.694 ns</td>
<td style="text-align: right">0.0076</td>
<td style="text-align: right">32 B</td>
</tr>
<tr>
<td>YieldingEnumerableAnyMethod</td>
<td style="text-align: right">32.55 ns</td>
<td style="text-align: right">8.911 ns</td>
<td style="text-align: right">0.488 ns</td>
<td style="text-align: right">0.0076</td>
<td style="text-align: right">32 B</td>
</tr>
<tr>
<td>NonYieldingEnumerableCountMethod</td>
<td style="text-align: right">18.03 ns</td>
<td style="text-align: right">9.068 ns</td>
<td style="text-align: right">0.497 ns</td>
<td style="text-align: right">0.0095</td>
<td style="text-align: right">40 B</td>
</tr>
<tr>
<td>NonYieldingEnumerableAnyMethod</td>
<td style="text-align: right">24.26 ns</td>
<td style="text-align: right">123.983 ns</td>
<td style="text-align: right">6.796 ns</td>
<td style="text-align: right">0.0095</td>
<td style="text-align: right">40 B</td>
</tr>
<tr>
<td>ListCountMethod</td>
<td style="text-align: right">71.39 ns</td>
<td style="text-align: right">187.181 ns</td>
<td style="text-align: right">10.260 ns</td>
<td style="text-align: right">0.0324</td>
<td style="text-align: right">136 B</td>
</tr>
<tr>
<td>ListCountProperty</td>
<td style="text-align: right">63.02 ns</td>
<td style="text-align: right">152.673 ns</td>
<td style="text-align: right">8.369 ns</td>
<td style="text-align: right">0.0324</td>
<td style="text-align: right">136 B</td>
</tr>
<tr>
<td>ListAnyMethod</td>
<td style="text-align: right">65.34 ns</td>
<td style="text-align: right">25.340 ns</td>
<td style="text-align: right">1.389 ns</td>
<td style="text-align: right">0.0324</td>
<td style="text-align: right">136 B</td>
</tr>
</tbody>
</table>
<p>As you can see from the above, the belief that Any() is faster than Count() for IEnumerables and Count is marginally faster than Any() and Count() is correct.</p>
<p>Yes there is a difference, but it is fairly inconsequential. Personally, I prefer the readability of Any() over Count > 0, so I will happily take the 2ns hit for spending a few milliseconds less reading the code.</p>
<p>Benchmarking code used to be something that took a great deal of effort, now it is so easy, next time you wonder if X is faster than Y, why not measure it?</p>
<p>I’ve not done a lot of benchmarking so if there are any flaws in the above that may be rendering the results inaccurate, please leave a comment.</p>Alan ParrWithin any community, there are often a series of untested truths. Within development communities, this often takes the form of X is faster than Y or Z is more efficient than W. While these are often true at some point, we don’t often test them to see if they remain true years later.My approach to improving a messy codebase.2021-12-31T00:00:00+00:002021-12-31T00:00:00+00:00http://alanparr.github.io/approach-to-improving-code<p>In the just over a decade I have been a professional developer, I’ve had to work with alot of messy code, only some of which I wrote. As a result, I’ve spent alot of time refactoring messy code to try and make it in to something better and want to share some of my methods for achieving this.</p>
<p>How I approach this has changed over the years but the below are areas I try to focus on and why. The order I do them in and how hard I go on them depends on the code base.</p>
<h2 id="how-did-this-happen">How did this happen?</h2>
<p>First, we need to understand how messy and poorly structured code comes in to existence in the first place.</p>
<h3 id="inexperienced-developers">Inexperienced developers</h3>
<p>One or more inexperienced devs with little or no experience were given a task and likely told to get on with it with no supervision and no experienced members to point them in the right direction. What results is a code-base that probably does the job, but is brittle and hard to change.</p>
<h3 id="lack-of-standards-andor-conventions">Lack of standards and/or conventions</h3>
<p>Give 10 developers a task and they will likely all implement it differently and all of them will probably be right in one way or another. Compound this over time in a team and you’ll end up with an inconsistent code base with no defined architecture where individual pieces integrate poorly or not at all. One person’s <code class="language-plaintext highlighter-rouge">Provider</code> is another person’s <code class="language-plaintext highlighter-rouge">Manager</code>, which to another person is a <code class="language-plaintext highlighter-rouge">Factory</code>. Mix and match these terms and it becomes impossible to determine how any individual component of the system is going to behave.</p>
<h3 id="lack-of-ownership-aka-just-get-it-done">Lack of ownership (AKA just get it done)</h3>
<p>This is similar to the previous item but is very much a function of developers being used as resources with no ownership over the codebase they work on and no power to define the priorities of what happens to it, all the while being under pressure to deliver more and more complex features ever faster than before.</p>
<p>When this happens over a prolonged period of time, the code base will become messier and no matter how much they shout about technical debt and the need to improve it, the business will push that back to <code class="language-plaintext highlighter-rouge">the next sprint</code> or <code class="language-plaintext highlighter-rouge">after this feature is done</code>, developer turnover will likely increase, and the business will be faced with slowing down development, reducing quality, or increasing the number of devs working on features to compensate for how difficult it is now to make changes. But this will not scale over time.</p>
<p>I’m sure there are other reasons I haven’t listed, but these are the big ones I have encountered personally.</p>
<p>Whatever the reason, you’ve been left with an unholy mess that you not only need to understand but also be able to fix and change.</p>
<h2 id="where-to-start">Where to start?</h2>
<p>So you’ve got this messy code-base in front of you and we’ll assume you’re new to it, so now you’ve got to figure out how to make it better in small ways that you can fit in between features. This is what I do.</p>
<p>The conventional wisdom will generally be “put tests in before touching anything”. This advice is absolutely correct, but in reality, there are things you can do that are relatively low in risk without having test coverage.</p>
<p>I’m not going to go over how to restructure the code as there is already plenty of material available on how to do this. I want to talk more about the things that often don’t get mentioned - tidying up the code as a precursor to performing massive life-altering refactorings.</p>
<h2 id="but-why">But why?</h2>
<p>Old code-bases can be messy. They’ve seen many different <code class="language-plaintext highlighter-rouge">best practice</code> implementations which have changed over time. They’ve got dead-code, zombie-code, code where only the developer who wrote it and god knew how it worked and now only god knows.</p>
<p>Refactoring is the task of taking this code and transforming it in to something better, but that is not an overnight task nor a straight path.</p>
<p>I believe there is a great deal of value in performing simple tidying up of the code before you go to the extent of refactoring it, especially if you have to carry on changing it in the meantime.</p>
<p>It is akin to buying a house with a badly overgrown garden. You know you are probably going to rip it all out at some point, but for the near-term, you want to trim it all back, remove the rotten shed and the three mouldy seats taken from a 1992 Ford Mondeo that you found down behind the garage, so you can get an idea of what you’ve actually got to work with once all the cruft is removed.</p>
<h3 id="ide-warnings">IDE Warnings</h3>
<p>Every IDE will raise warnings, and they do this for a good reason. If you’ve got a messy code-base, chances are you’ve got a boatload of warnings. Some of these may be small problems like unused variables or larger issues like the IDE trying to warn you about code that will actually cause problems at runtime, such as mismatched assembly versions or suspicious looking code blocks.</p>
<p>In Visual Studio, and I’m sure in other IDEs, each warning has a unique code and will likely provide a link to a page explaining the reason for the warning and sometimes how to fix it.</p>
<p>You’ll never see any of this, of course, as the warnings you are actually interested in become the proverbial needle in the haystack and the count in the warnings window is a daily reminder of how much the code-base sucks.</p>
<p>The first thing you can do is start reducing these. This is my strategy:</p>
<ul>
<li>Pick a warning code and filter by that code in the IDE.</li>
<li>Start a branch, run through the list and fix each one. If you have a build server, commit and push it up for a build regularly.</li>
<li>Once done, merge the branch and move on to the next one.</li>
</ul>
<p>One thing to note about this strategy is that while it will get rid of the warnings quickly, you end up touching a lot of files. To keep the chances of merge conflicts low, no individual branch should last longer than a day, two at most. This may mean you need multiple branches to tackle all the warnings for a single code, but branches are cheap and it will be better in the long run to not have a massive branch with hundreds of files in it to merge after 2 weeks of the rest of your team making changes, that way lies madness and bugs.</p>
<p>Another thing to note is that, while this is generally low-risk as fixing warnings will mostly not change logic, there is always a possibility of unexpected mutations, so keep your wits about you and exercise your own judgement on which warnings you fix and which you maybe leave until later, possibly until you’ve got time to improve unit test coverage around the code in question.</p>
<p>Reducing (or removing) warnings will have a number of benefits:</p>
<ol>
<li>The warnings list becomes relevant again, so devs can use it for its intended purpose of the IDE warning them when they have potentially introduced an issue.</li>
<li>Working on a code-base that had 7k+ warnings and taking that down to 3k (still a way to go) had the very pleasing side-effect of improving performance of Visual Studio. It wasn’t life changing, but it has been noticeable for me.</li>
<li>If you’re new to the code-base, tackling the warnings will take you all over the place so it is a great way to familiarise yourself with the code. This increased knowledge will pay dividends in the future.</li>
</ol>
<p>As an added bonus, when you’re jumping around the code-base checking out these warnings, you may find code that appears to be obsolete or unused. Good examples of this are methods that are no-ops or have had their contents commented out (yes, this is a thing) or methods whose name suggests they have a side-effect (such as CreateFoo) yet they don’t contain any code that would appear to achieve that objective. Where you see these, take some time to check out their usages and, if they appear to be dead code, mark then with the <code class="language-plaintext highlighter-rouge">System.Obsolete</code> attribute or at least a <code class="language-plaintext highlighter-rouge">//TODO:</code> so you can come back to them later to validate if they really are dead code or maybe just badly named.</p>
<h3 id="obsolete-or-unused-code-code">Obsolete or unused code code</h3>
<p>Code that is marked with the <code class="language-plaintext highlighter-rouge">System.Obsolete</code> attribute will appear in warnings, but this is worth discussing on its own.</p>
<p>A code-base that has not been properly looked after over time will likely have an abundance of unused and obsolete code. If you’re lucky it will have been marked with the obsolete attribute. Check these out see what you can safely remove.</p>
<p>Removing the obsolete code has an obvious benefit of reducing the size of the code base and therefore opportunities for things to break. Obsolete code is often an old version of something that was rewritten but never fully migrated to, so you’ll probably find you’re doing the same thing multiple different ways in multiple different places. Reducing this duplication will improve your stability and also ensure you’re not using this obsolete code, which was probably obsolete for a reason.</p>
<p>As you delete obsolete code, you’ll often find more code that was only used by the now-deleted obsolete code that may also be able to be deleted. As a result, it may take a few passes to get it all. IDEs can be great aids in helping you find code that can be be safely deleted, but they are not infallible so look upon suggestions with some suspicion and try to validate code is actually obsolete/unused before you delete it.</p>
<p>This is much easier in a statically-typed, compiled language such as C# where the compiler will make you very aware if you’ve deleted something that is used. It is harder with interpreted languages but there are often tools available that will provide similar functionality.</p>
<p>One example that caught me out was on a large code-base with a lot of partial classes. Visual Studio would often flag up variables within these classes as not used, however it had managed to miss usage in another partial implementation of that class. Commit and build regularly and you can keep the chances of this happening low.</p>
<p>If you find obsolete code that cannot currently be removed, for example an old version of something that has been re-implemented elsewhere but is still in use in too many places to fix currently, use the <code class="language-plaintext highlighter-rouge">System.Obsolete</code> attribute to mark the code to come back to later and use the description attribute to explain why you think it is obsolete. It is extremely frustrating to find a piece of code with the Obsolete attribute on it but nothing to explain <em>why</em> it is obsolete.</p>
<h2 id="nuget">Nuget</h2>
<p>This covers 2 issues that are often found in older code bases.</p>
<ul>
<li>Nuget packages that are massively out of date.</li>
<li>Checked in assemblies from the days before Nuget existed.</li>
</ul>
<p>It is good practice to keep your dependencies up to date for obvious reasons such as security, but also to get performance improvements and new features. The cost of updating your dependencies will likely be lower if you’re doing it regularly than if you try to jump several years worth of major versions in one go.</p>
<p>Before Nuget came along in 2010, we had to keep our dependencies somewhere, and that was often a folder, usually in the root of a repository, called “Third Party”, “reference assemblies” or something similar.</p>
<p>If the code-base pre-dates Nuget and hasn’t been well maintained, it will likely have a folder like this and you should seek to eliminate it.</p>
<p>These old dependencies will be out of date and therefore potentially a security risk. They are also a ticking time bomb, waiting for the day when you want to change framework versions or make some other changes that will break them and leave you in a pickle.</p>
<p>If they exist in Nuget, it should be a simple case of referencing that instead and getting rid of your copy. If you’re lucky, the same version may even be available as a quick win if you don’t want to risk updating right now.</p>
<p>If you can’t find a dependency on Nuget then it’s time to either write something that does the same job or search Nuget for a current library that will fill the same role.</p>
<p>You’ll have to do this eventually so you may as well do it at a time of your choosing rather than when a new feature or bug fix forces your hand.</p>
<h2 id="modernise-the-style">Modernise the style</h2>
<p>You’ve probably got a lot of code that is on the older end of the spectrum and doesn’t conform to modern standards or make use of modern language features.</p>
<p>This can result in the IDE seemingly assaulting you with so many code improvement suggestions that you can’t see the wood for the trees.</p>
<p>While modernising the style of the code may seem like a trivial matter that we really shouldn’t worry about, these new language features were introduced for a reason, often to improve readability, and in our messy code-base, we could use every bit of readability we can get.</p>
<p>To see how much of a difference this makes in your case, pick any source file, preferrably one that is small but tough to understand. Then start following the IDE recommendations.</p>
<ul>
<li>Convert explicit typing to implicit typing with var.</li>
<li>Convert uses of <code class="language-plaintext highlighter-rouge">String</code> or <code class="language-plaintext highlighter-rouge">Int32</code> to their keyword equivalents of <code class="language-plaintext highlighter-rouge">string</code> and <code class="language-plaintext highlighter-rouge">int</code>.</li>
<li>Rename variables to match an agreed upon convention, either using the language default or agree one within your team and use the IDE to automate implementing it.</li>
<li>Remove unnecessary blank lines between methods, variables, and classes.</li>
<li>Remove unused usings.</li>
<li>Remove <code class="language-plaintext highlighter-rouge">this</code>, renaming variables where necessary to remove the need for it.</li>
</ul>
<p>Try every recommendation, even if you’re not sure about it. Don’t be afraid to reverse it if you think it makes the code look worse.</p>
<p>Once you’ve done all this, you may well find that the readability of this file has just increased massively, or you may find it has made no difference at all other than to reduce how much it upsets the IDE, your mileage may vary but give it a go and see if it makes a material difference in your code.</p>
<h2 id="testing">Testing</h2>
<p>At what point you start to focus on this really depends on your appetite for risk.</p>
<p>You might make this your first stop before touching anything or you may have the confidence that you can leave it until you have tidied things up a bit, that one is down to you.</p>
<p>Most of the tasks I have listed thus far can be done without test coverage with relatively minimal risk.</p>
<p>At some point though, once you’ve tidied up, you’re going to have to start making higher risk changes that you just don’t want to attempt without any test coverage.</p>
<p>Writing tests on a new code-base is usually very simple, but on an older one, it can be disproportionately difficult to get in to a position where you can test even the smallest piece of functionality and there is a whole spectrum between those two options, so now you need to find out where you sit on that spectrum.</p>
<p>Pick a simple case as a canary to set how easy it is to create tests for existing code and just starting writing. First try to get to a point where you can call the method without it failing or instantiate the class without it throwing an exception.</p>
<p>This will often expose things like hard dependencies on external services or subsystems, such as databases and file systems. It will also begin to show you the hidden assumptions that exist within the application.</p>
<p>In the first instance, try to find ways to provide these dependencies without making changes to the code. This should be relatively easy if you have Dependency Injection, but may require you to be <a href="/dependency-substitution-for-legacy-code">creative</a> if you haven’t.</p>
<p>If you need to make changes, try to keep them to a minimum and introduce as little extra logic in to the calling code as possible, we don’t want to make the mess any worse.</p>
<p>Factories are often a good choice here as they can generally slip in with relatively little ceremony. They likely won’t be the end-game but can be a useful aide in your journey.</p>
<p>You may sometimes have to introduce abstractions that you don’t really like in order to unblock a piece of code and allow you to move on to the next stage of improvement. Don’t be afraid to do this as a step on the path from where you are to where you want to be.</p>
<p>Once you’ve got to the point where you can call your method or instantiate your class without exception and get some asserts in there, you’ve just created a foundation that you can build upon to improve the coverage on your code-base and start testing more and more complicated parts of the system.</p>
<p>I don’t want to go too deeply in to the finer points of refactoring as there is a great deal of information out there written by people who know far more and are significantly better at communicating that knowledge than I am, however one book I can recommend is <a href="https://smile.amazon.co.uk/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052/ref=sr_1_1?keywords=working+effectively+with+legacy+code&qid=1640980440&sprefix=working+effe%2Caps%2C73&sr=8-1">Working Effectively with Legacy Code</a> by Michael Feathers, I found it very useful to fill in some gaps in my knowledge and inform my approach as i’ve embarked on refactorings in the past.</p>
<p>I hope the above was helpful. If you’ve got a tip or trick that works for you, feel free to post it in a comment.</p>Alan ParrIn the just over a decade I have been a professional developer, I’ve had to work with alot of messy code, only some of which I wrote. As a result, I’ve spent alot of time refactoring messy code to try and make it in to something better and want to share some of my methods for achieving this.Handling rate limiting with RestSharp and Polly2021-12-22T00:00:00+00:002021-12-22T00:00:00+00:00http://alanparr.github.io/handle-rate-limiting-restsharp-polly<p>I was recently working on an integration with an API that has Rate-Limiting, that is if you go over the permitted number of requests within a certain period of time, you’ll receive a <code class="language-plaintext highlighter-rouge">429</code> response along with a header telling you how long you need to wait before trying again.</p>
<p>I’ve been watching Polly for years but haven’t had the chance to use it, so this seemed like the perfect opportunity and it was really quite simple:</p>
<p>Below is a method that returns a policy, which is what Polly uses for a lot of it’s retry functionality.</p>
<p>This takes a delegate which returns an <code class="language-plaintext highlighter-rouge">IRestResponse</code>. It checks the StatusCode property of that reponse and, if it is <code class="language-plaintext highlighter-rouge">429</code>, it enters the <code class="language-plaintext highlighter-rouge">sleepDurationProvider</code> method.</p>
<p>In my case, this uses a class called RateStatistics that takes data from the headers of each response, it then returns either the value that was provided by the response of defaults to 5 seconds if it wasn’t. I don’t expect this will ever happen but wanted to be belt and braces on it and figured 5 seconds is a good number that should cover us in most cases.</p>
<p>It is worth noting that this request is being sent from a background service so there is no user sitting waiting for this request to finished, which would obviously affect our choice of 5 seconds as the fallback value.</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">private</span> <span class="n">RetryPolicy</span><span class="p"><</span><span class="n">IRestResponse</span><span class="p">></span> <span class="nf">GetPolicy</span><span class="p">()</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">policy</span> <span class="p">=</span> <span class="n">Policy</span><span class="p">.</span><span class="n">HandleResult</span><span class="p"><</span><span class="n">IRestResponse</span><span class="p">>(</span><span class="n">p</span> <span class="p">=></span> <span class="p">(</span><span class="kt">int</span><span class="p">)</span><span class="n">p</span><span class="p">.</span><span class="n">StatusCode</span> <span class="p">==</span> <span class="m">429</span><span class="p">).</span><span class="nf">WaitAndRetry</span><span class="p">(</span>
<span class="n">retryCount</span><span class="p">:</span> <span class="m">3</span><span class="p">,</span>
<span class="n">sleepDurationProvider</span><span class="p">:</span> <span class="p">(</span><span class="n">retryCount</span><span class="p">,</span> <span class="n">response</span><span class="p">,</span> <span class="n">context</span><span class="p">)</span> <span class="p">=></span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">stats</span> <span class="p">=</span> <span class="n">RateStatistics</span><span class="p">.</span><span class="nf">FromRestsharpResponse</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">Result</span><span class="p">);</span>
<span class="kt">var</span> <span class="n">msToWait</span> <span class="p">=</span> <span class="n">stats</span><span class="p">.</span><span class="n">TimeUntilWindowResetMilliseconds</span> <span class="p">??</span> <span class="m">5000</span><span class="p">;</span>
<span class="k">return</span> <span class="n">TimeSpan</span><span class="p">.</span><span class="nf">FromMilliseconds</span><span class="p">(</span><span class="n">msToWait</span><span class="p">);</span>
<span class="p">},</span>
<span class="n">onRetry</span><span class="p">:</span> <span class="p">(</span><span class="n">response</span><span class="p">,</span> <span class="n">timespan</span><span class="p">,</span> <span class="n">retryCount</span><span class="p">,</span> <span class="n">context</span><span class="p">)</span> <span class="p">=></span>
<span class="p">{</span>
<span class="c1">// Logging so we know the retry has happened.</span>
<span class="n">_logger</span><span class="p">.</span><span class="nf">Log</span><span class="p">(</span><span class="n">Level</span><span class="p">.</span><span class="n">Warning</span><span class="p">,</span> <span class="s">$"Retry attempt #</span><span class="p">{</span><span class="n">retryCount</span><span class="p">}</span><span class="s"> for </span><span class="p">{</span><span class="n">response</span><span class="p">.</span><span class="n">Result</span><span class="p">.</span><span class="n">ResponseUri</span><span class="p">}</span><span class="s"> due to rate-limiting."</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">);</span>
<span class="k">return</span> <span class="n">policy</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">RateStatistics</code> class is fairly simple. It just receives an IRestResponse and pulls the values we are interested in out of the headers. For the sake of clarity, i’ve stripped out any properties/values that aren’t relevant to this post.</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">internal</span> <span class="k">class</span> <span class="nc">RateStatistics</span>
<span class="p">{</span>
<span class="k">public</span> <span class="kt">int</span><span class="p">?</span> <span class="n">TimeUntilWindowResetMilliseconds</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="k">set</span><span class="p">;</span> <span class="p">}</span>
<span class="k">public</span> <span class="k">static</span> <span class="n">RateStatistics</span> <span class="nf">FromRestsharpResponse</span><span class="p">(</span><span class="n">IRestResponse</span> <span class="n">response</span><span class="p">)</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">windowTimeRemaining</span> <span class="p">=</span> <span class="n">response</span><span class="p">.</span><span class="n">Headers</span><span class="p">.</span><span class="nf">FirstOrDefault</span><span class="p">(</span><span class="n">x</span> <span class="p">=></span> <span class="n">x</span><span class="p">.</span><span class="n">Name</span> <span class="p">==</span> <span class="s">"X-Rate-Limit-Time-Reset-Ms"</span><span class="p">)?.</span><span class="n">Value</span><span class="p">;</span>
<span class="k">return</span> <span class="k">new</span> <span class="n">RateStatistics</span>
<span class="p">{</span>
<span class="n">TimeUntilWindowResetMilliseconds</span> <span class="p">=</span> <span class="n">windowTimeRemaining</span> <span class="p">==</span> <span class="k">null</span> <span class="p">?</span> <span class="p">(</span><span class="kt">int</span><span class="p">?)</span><span class="k">null</span> <span class="p">:</span> <span class="kt">int</span><span class="p">.</span><span class="nf">Parse</span><span class="p">(</span><span class="n">windowTimeRemaining</span><span class="p">.</span><span class="nf">ToString</span><span class="p">()),</span>
<span class="p">};</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Our code to call our RestSharp client with our retry policy is really simple:</p>
<p>Before adding Polly:</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">result</span> <span class="p">=</span> <span class="n">client</span><span class="p">.</span><span class="nf">Execute</span><span class="p">(</span><span class="n">request</span><span class="p">);</span>
</code></pre></div></div>
<p>After adding Polly:</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">result</span> <span class="p">=</span> <span class="nf">GetPolicy</span><span class="p">().</span><span class="nf">Execute</span> <span class="p">(()</span> <span class="p">=></span> <span class="n">client</span><span class="p">.</span><span class="nf">Execute</span><span class="p">(</span><span class="n">request</span><span class="p">));</span>
</code></pre></div></div>
<p>Now if we receive a <code class="language-plaintext highlighter-rouge">429</code> response, Polly will wait the amount of time provided by the API we are calling and will try up to 3 times before allowing the request to fail.</p>Alan ParrI was recently working on an integration with an API that has Rate-Limiting, that is if you go over the permitted number of requests within a certain period of time, you’ll receive a 429 response along with a header telling you how long you need to wait before trying again.Bicep - Create Azure Service Bus with topic and filtered subscription.2021-11-14T00:00:00+00:002021-11-14T00:00:00+00:00http://alanparr.github.io/bicep-create-servicebus-with-topic-and-subscription<p>It’s been a while since I’ve done anything with Service Bus so to get myself back in to it and indulge my current enjoyment of Bicep, I thought i’d start with creating the service bus, along with a topic, and a filtered subscription, all in Bicep.</p>
<p>To start with, the main.bicep which is virtually the same as last time you saw it:</p>
<pre><code class="language-bicep">param appname string = 'testapp'
param environment string = 'staging'
param region string = 'ukwest'
targetScope = 'subscription'
//Create the resource group.
resource sa 'Microsoft.Resources/resourceGroups@2021-01-01' = {
name: 'rg-${appname}-${environment}'
location: region
}
//Run the storage module, setting scope to the resource group we just created.
module res './servicebus.bicep' = {
name: 'resourceDeploy'
params: {
appname: appname
envtype: environment
}
scope: resourceGroup(sa.name)
}
</code></pre>
<p>I’ve just changed it to call <code class="language-plaintext highlighter-rouge">servicebus.bicep</code> instead of <code class="language-plaintext highlighter-rouge">storage.bicep</code>.</p>
<p>The servicebus.bicep consists of the following:</p>
<pre><code class="language-bicep">targetScope = 'resourceGroup'
param appname string
param envtype string
resource servicebus 'Microsoft.ServiceBus/namespaces@2021-06-01-preview' = {
name: 'sb${appname}${envtype}'
location: resourceGroup().location
sku: {
name: 'Standard'
tier: 'Standard'
}
}
</code></pre>
<p>Start with defining our parameters that we will take in from the main.bicep and creating the Service Bus namespace. This was pretty straight forward, using the <a href="https://docs.microsoft.com/en-us/azure/templates/microsoft.servicebus/namespaces?tabs=bicep">docs</a> to get the parameter names.</p>
<pre><code class="language-bicep">resource orders 'Microsoft.ServiceBus/namespaces/topics@2021-06-01-preview' = {
name: 'orders'
parent: servicebus
properties: {
defaultMessageTimeToLive: 'P6M' //ISO 8601
status: 'Active'
}
}
</code></pre>
<p>Next we create the topic. I did get stuck here for a bit as it wasn’t obvious what the value of parent should be. Eventually, I realised it was the resource object for the <code class="language-plaintext highlighter-rouge">servicebus</code> itself, which is handy to know as it looks like <code class="language-plaintext highlighter-rouge">parent</code> is used in various places in Bicep.</p>
<p>The <code class="language-plaintext highlighter-rouge">defaultMessageTimeToLive</code> is using ISO 8601 duration format which is a fairly simple format to understand, brief explanation is <a href="https://www.digi.com/resources/documentation/digidocs/90001437-13/reference/r_iso_8601_duration_format.htm#:~:text=Represents%20a%20duration%20of%20three%20years%2C%20six%20months%2C,duration%20formatupdated%20on%2020%20Sep%202019%2010%3A14%20AM">here</a> and there is a calculator <a href="https://www.345tool.com/time-util/time-duration-calculator">here</a>.</p>
<pre><code class="language-bicep">resource amazonpushlistingrule 'Microsoft.ServiceBus/namespaces/topics/subscriptions/rules@2021-06-01-preview' = {
name: 'operationtype'
parent: orderreceived
properties: {
filterType: 'CorrelationFilter'
correlationFilter: {
properties: {
'operationtype':'orderreceived'
}
}
}
}
</code></pre>
<p>Next up is the subscription. Subscriptions can have filters that limit the messages that go in to that subscription from the topic, in this case I’m looking for a property called <code class="language-plaintext highlighter-rouge">operationtype</code> to have value of <code class="language-plaintext highlighter-rouge">orderreceived</code>. Only messages where this is true will appear in this subscription.</p>
<pre><code class="language-bicep">resource catchall 'Microsoft.ServiceBus/namespaces/topics/subscriptions@2021-06-01-preview' = {
name: 'catchall'
parent: orders
properties: {
deadLetteringOnMessageExpiration: true
defaultMessageTimeToLive: 'P6M'
lockDuration: 'PT5M'
maxDeliveryCount: 20
status: 'Active'
}
}
</code></pre>
<p>Last, but not least, I create a catchall subscription with no filter, so if any messages don’t end up where I expect them to, I can look at them in this subscription to see what was wrong.</p>
<p>Then I just deploy using the below Az Cli command</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>az deployment sub create -f .\main.bicep -l ukwest
</code></pre></div></div>
<p>For easy copy/paste, the full content of servicebus.bicep is below:</p>
<pre><code class="language-bicep">targetScope = 'resourceGroup'
param appname string
param envtype string
resource servicebus 'Microsoft.ServiceBus/namespaces@2021-06-01-preview' = {
name: 'sb${appname}${envtype}'
location: resourceGroup().location
sku: {
name: 'Standard'
tier: 'Standard'
}
}
resource orders 'Microsoft.ServiceBus/namespaces/topics@2021-06-01-preview' = {
name: 'orders'
parent: servicebus
properties: {
defaultMessageTimeToLive: 'P6M' //ISO 8601
status: 'Active'
}
}
resource orderreceived 'Microsoft.ServiceBus/namespaces/topics/subscriptions@2021-06-01-preview' = {
name: 'orderreceived'
parent: orders
properties: {
deadLetteringOnMessageExpiration: true
defaultMessageTimeToLive: 'P6M'
lockDuration: 'PT5M'
maxDeliveryCount: 20
status: 'Active'
}
}
resource amazonpushlistingrule 'Microsoft.ServiceBus/namespaces/topics/subscriptions/rules@2021-06-01-preview' = {
name: 'operationtype'
parent: orderreceived
properties: {
filterType: 'CorrelationFilter'
correlationFilter: {
properties: {
'operationtype':'orderreceived'
}
}
}
}
resource catchall 'Microsoft.ServiceBus/namespaces/topics/subscriptions@2021-06-01-preview' = {
name: 'catchall'
parent: orders
properties: {
deadLetteringOnMessageExpiration: true
defaultMessageTimeToLive: 'P6M'
lockDuration: 'PT5M'
maxDeliveryCount: 20
status: 'Active'
}
}
</code></pre>Alan ParrIt’s been a while since I’ve done anything with Service Bus so to get myself back in to it and indulge my current enjoyment of Bicep, I thought i’d start with creating the service bus, along with a topic, and a filtered subscription, all in Bicep.Dependency substitution for legacy code2021-09-17T00:00:00+00:002021-09-17T00:00:00+00:00http://alanparr.github.io/dependency-substitution-for-legacy-code<p>This post shows an example of how to introduced a form of dependency injection in to legacy code with minimal changes.
I refer to this as dependency substitution because, in my view at least, there is a subtle difference between this and depdendency injection.</p>
<p>With dependency injection, we are <em>injecting</em> the dependency via either constructor or property injection. The dependency is set once by our IOC container to whatever type that dependency is set to be set to.</p>
<p>With dependency substitution, we’re performing a bait and switch, swapping out the previously hard-coded dependency via a surreptitously snuck in substitute.</p>
<p>It’s a small difference admittedly, but as this technique is very much intended as a temporary solution to get tests around your legacy code so it can be modernised safely, I didn’t want to give the impression that this technique ticks the <code class="language-plaintext highlighter-rouge">we've added dependency injection</code> checkbox because in my view it doesn’t.</p>
<p>Anyway, on to the code. It is a slightly convoluted example but hopefully demonstrates the technique.</p>
<p>All the code is available on <a href="https://github.com/AlanParr/LegacyAppTesting">Github</a>.</p>
<h2 id="before">Before</h2>
<p>Program.cs - This instantiates a BoredomSuggestionService, asks the user if they’re bored and returns an activity to do until they answer with <code class="language-plaintext highlighter-rouge">N</code></p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">using</span> <span class="nn">System</span><span class="p">;</span>
<span class="k">using</span> <span class="nn">System.Threading.Tasks</span><span class="p">;</span>
<span class="k">namespace</span> <span class="nn">LegacyAppTesting.ConsoleApp</span>
<span class="p">{</span>
<span class="k">internal</span> <span class="k">class</span> <span class="nc">Program</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">static</span> <span class="k">async</span> <span class="n">Task</span> <span class="nf">Main</span><span class="p">(</span><span class="kt">string</span><span class="p">[]</span> <span class="n">args</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">//Instantiate bored client.</span>
<span class="kt">var</span> <span class="n">service</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">BoredomSuggestionService</span><span class="p">();</span>
<span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">"Bored? Would you like a suggestion for something to do? (Y/N): "</span><span class="p">);</span>
<span class="k">while</span> <span class="p">(</span><span class="n">Console</span><span class="p">.</span><span class="nf">ReadLine</span><span class="p">()</span> <span class="p">==</span> <span class="s">"Y"</span><span class="p">)</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">randomActivity</span> <span class="p">=</span> <span class="k">await</span> <span class="n">service</span><span class="p">.</span><span class="nf">FindSomethingToDo</span><span class="p">();</span>
<span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">" How about "</span> <span class="p">+</span> <span class="n">randomActivity</span><span class="p">.</span><span class="n">Activity</span> <span class="p">+</span> <span class="s">"?"</span><span class="p">);</span>
<span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">""</span><span class="p">);</span>
<span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">"Still bored?"</span><span class="p">);</span>
<span class="p">}</span>
<span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">"So glad you're not bored any more, enjoy your activity!"</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The BoredomSuggestionService is quite simple:</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">class</span> <span class="nc">BoredomSuggestionService</span>
<span class="p">{</span>
<span class="k">private</span> <span class="n">BoredClient</span> <span class="n">_client</span><span class="p">;</span>
<span class="k">public</span> <span class="nf">BoredomSuggestionService</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">_client</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">BoredClient</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">async</span> <span class="n">Task</span><span class="p"><</span><span class="n">BoredResponse</span><span class="p">></span> <span class="nf">FindSomethingToDo</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">return</span> <span class="k">await</span> <span class="n">_client</span><span class="p">.</span><span class="nf">FindSomethingToDo</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>It instantiates a BoredClient which talks to the API at <a href="https://www.boredapi.com">BoredAPI.com</a> to get an activity suggestion.</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">public</span> <span class="k">class</span> <span class="nc">BoredClient</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">async</span> <span class="n">Task</span><span class="p"><</span><span class="n">BoredResponse</span><span class="p">></span> <span class="nf">FindSomethingToDo</span><span class="p">()</span>
<span class="p">{</span>
<span class="c1">//Get activity.</span>
<span class="kt">var</span> <span class="n">client</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">HttpClient</span><span class="p">();</span>
<span class="kt">var</span> <span class="n">result</span> <span class="p">=</span> <span class="k">await</span> <span class="n">client</span><span class="p">.</span><span class="nf">GetAsync</span><span class="p">(</span><span class="s">"http://www.boredapi.com/api/activity/"</span><span class="p">);</span>
<span class="kt">var</span> <span class="n">bodyString</span> <span class="p">=</span> <span class="k">await</span> <span class="n">result</span><span class="p">.</span><span class="n">Content</span><span class="p">.</span><span class="nf">ReadAsStringAsync</span><span class="p">();</span>
<span class="k">return</span> <span class="n">Newtonsoft</span><span class="p">.</span><span class="n">Json</span><span class="p">.</span><span class="n">JsonConvert</span><span class="p">.</span><span class="n">DeserializeObject</span><span class="p"><</span><span class="n">BoredResponse</span><span class="p">>(</span><span class="n">bodyString</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<h2 id="after">After</h2>
<p>What we are going to do is modify BoredomSuggestionService so we can sneak in a mocked version of BoredClient so we can test it.</p>
<p>We’ll go step-by-step over the changes needed to facilitate this.</p>
<p>First, we create an interface for <code class="language-plaintext highlighter-rouge">BoredClient</code></p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">public</span> <span class="k">interface</span> <span class="nc">IBoredClient</span>
<span class="p">{</span>
<span class="n">Task</span><span class="p"><</span><span class="n">BoredResponse</span><span class="p">></span> <span class="nf">FindSomethingToDo</span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Then we create a really simple factory:</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">public</span> <span class="k">class</span> <span class="nc">BoredClientFactory</span>
<span class="p">{</span>
<span class="c1">//Static override instance.</span>
<span class="k">private</span> <span class="k">static</span> <span class="n">IBoredClient</span> <span class="n">_overriddenClient</span><span class="p">;</span>
<span class="c1">//Protected so only available to classes that inherit from this one.</span>
<span class="k">protected</span> <span class="k">void</span> <span class="nf">SetClientInternalForTestingOnly</span><span class="p">(</span><span class="n">IBoredClient</span> <span class="n">boredClient</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">_overriddenClient</span> <span class="p">=</span> <span class="n">boredClient</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">public</span> <span class="n">IBoredClient</span> <span class="nf">GetClient</span><span class="p">()</span>
<span class="p">{</span>
<span class="c1">//Return either our overridden instance or instantiate a new one as we were before.</span>
<span class="k">return</span> <span class="n">_overriddenClient</span> <span class="p">??</span> <span class="k">new</span> <span class="nf">BoredClient</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This is the key to getting the mock in. There is a protected method that will allow setting the static overridden <code class="language-plaintext highlighter-rouge">IBoredClient</code>.
This method is only for testing and is named as such to make it clear to anyone coming across this in the future that they shouldn’t use it in Production.</p>
<p>We use the factory in <code class="language-plaintext highlighter-rouge">BoredomSuggestionService</code> as so, it is a very small change.</p>
<p>This</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">public</span> <span class="nf">BoredomSuggestionService</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">_client</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">BoredClient</span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div></div>
<p>changes to</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">public</span> <span class="nf">BoredomSuggestionService</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">_client</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">BoredClientFactory</span><span class="p">().</span><span class="nf">GetClient</span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="introducing-the-testing">Introducing the testing</h3>
<p>We’ve now made all the changes to production code that we need to.</p>
<p>Now we create a test project and in that project, create a class called <code class="language-plaintext highlighter-rouge">TestBoredClientFactory</code></p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">public</span> <span class="k">class</span> <span class="nc">TestBoredClientFactory</span><span class="p">:</span> <span class="n">BoredClientFactory</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">SetClient</span><span class="p">(</span><span class="n">IBoredClient</span> <span class="n">boredClient</span><span class="p">)</span>
<span class="p">{</span>
<span class="nf">SetClientInternalForTestingOnly</span><span class="p">(</span><span class="n">boredClient</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>and in our test we use this to get the <code class="language-plaintext highlighter-rouge">Mock<IBoredClient></code> in to the factory:</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="p">[</span><span class="n">Test</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">TestMockBoredClient</span><span class="p">()</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">mockClient</span> <span class="p">=</span> <span class="k">new</span> <span class="n">Mock</span><span class="p"><</span><span class="n">IBoredClient</span><span class="p">>();</span>
<span class="n">mockClient</span><span class="p">.</span><span class="nf">Setup</span><span class="p">(</span><span class="n">x</span> <span class="p">=></span> <span class="n">x</span><span class="p">.</span><span class="nf">FindSomethingToDo</span><span class="p">()).</span><span class="nf">Returns</span><span class="p">(</span><span class="n">Task</span><span class="p">.</span><span class="nf">FromResult</span><span class="p">(</span><span class="k">new</span> <span class="nf">BoredResponse</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">Activity</span> <span class="p">=</span> <span class="s">"Extreme Ironing"</span><span class="p">,</span>
<span class="n">Accessibility</span> <span class="p">=</span> <span class="m">0.1</span><span class="p">,</span>
<span class="n">Key</span> <span class="p">=</span> <span class="s">"EXTREME_IRONING"</span><span class="p">,</span>
<span class="n">Participants</span> <span class="p">=</span> <span class="m">1</span><span class="p">,</span>
<span class="n">Price</span> <span class="p">=</span> <span class="m">10</span><span class="p">,</span>
<span class="n">Type</span> <span class="p">=</span> <span class="s">"Extreme Sports"</span>
<span class="p">}));</span>
<span class="c1">//Setup</span>
<span class="k">new</span> <span class="nf">TestBoredClientFactory</span><span class="p">().</span><span class="nf">SetClient</span><span class="p">(</span><span class="n">mockClient</span><span class="p">.</span><span class="n">Object</span><span class="p">);</span>
<span class="kt">var</span> <span class="n">result</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">BoredomSuggestionService</span><span class="p">().</span><span class="nf">FindSomethingToDo</span><span class="p">().</span><span class="n">Result</span><span class="p">;</span>
<span class="n">Assert</span><span class="p">.</span><span class="nf">AreEqual</span><span class="p">(</span><span class="s">"Extreme Ironing"</span><span class="p">,</span> <span class="n">result</span><span class="p">.</span><span class="n">Activity</span><span class="p">);</span>
<span class="n">Assert</span><span class="p">.</span><span class="nf">AreEqual</span><span class="p">(</span><span class="s">"EXTREME_IRONING"</span><span class="p">,</span> <span class="n">result</span><span class="p">.</span><span class="n">Key</span><span class="p">);</span>
<span class="n">Assert</span><span class="p">.</span><span class="nf">AreEqual</span><span class="p">(</span><span class="s">"Extreme Sports"</span><span class="p">,</span> <span class="n">result</span><span class="p">.</span><span class="n">Type</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>
<p>By calling the <code class="language-plaintext highlighter-rouge">TestBoredClientFactory.SetClient</code> method, we can push the Mock in to the static <code class="language-plaintext highlighter-rouge">_overriddenClient</code> which will then be produced by the <code class="language-plaintext highlighter-rouge">BoredClientFactory</code> in <code class="language-plaintext highlighter-rouge">BoredomSuggestionService</code>.</p>
<p>As a result of this bait-and-switch we can now substitute the BoredClient with our Mock and get some tests in.</p>
<h2 id="conclusion">Conclusion</h2>
<p>I’ve used this technique a few times in cases where DI was not possible and I couldn’t risk making large-scale changes without getting at least some tests in place.</p>
<p>It works well, but to reiterate, if true dependency injection is an option that you have available to you, please do that instead.</p>
<p>I hope this is helpful, comments welcome as usual.</p>Alan ParrThis post shows an example of how to introduced a form of dependency injection in to legacy code with minimal changes. I refer to this as dependency substitution because, in my view at least, there is a subtle difference between this and depdendency injection.Copying messages between SQS queues2021-09-08T00:00:00+00:002021-09-08T00:00:00+00:00http://alanparr.github.io/copying-messages-between-sqs-queues<p>This week I had the need to copy a load of messages from one SQS queue to another, below is the body of the Linqpad script I used to do it.</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">void</span> <span class="nf">Main</span><span class="p">()</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">seenIds</span> <span class="p">=</span> <span class="k">new</span> <span class="n">List</span><span class="p"><</span><span class="kt">string</span><span class="p">>();</span>
<span class="kt">var</span> <span class="n">sidContainer</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">DumpContainer</span><span class="p">(</span><span class="s">"Seen Ids: "</span><span class="p">).</span><span class="nf">Dump</span><span class="p">();</span>
<span class="kt">var</span> <span class="n">sqsConfig</span> <span class="p">=</span> <span class="k">new</span> <span class="n">AmazonSQSConfig</span> <span class="p">{</span> <span class="n">ServiceURL</span> <span class="p">=</span> <span class="s">"AWS_REGION_URL"</span> <span class="p">};</span>
<span class="kt">var</span> <span class="n">client</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">AmazonSQSClient</span><span class="p">(</span><span class="s">"AWSKEY"</span><span class="p">,</span> <span class="s">"AWSSECRET"</span><span class="p">,</span> <span class="n">sqsConfig</span><span class="p">);</span>
<span class="kt">var</span> <span class="n">sourceQueueUrl</span> <span class="p">=</span> <span class="n">client</span><span class="p">.</span><span class="nf">GetQueueUrlAsync</span><span class="p">(</span><span class="s">"NAME_OF_SOURCE_QUEUE"</span><span class="p">).</span><span class="n">Result</span><span class="p">.</span><span class="n">QueueUrl</span><span class="p">.</span><span class="nf">Dump</span><span class="p">();</span>
<span class="kt">var</span> <span class="n">destinationQueueUrl</span> <span class="p">=</span> <span class="n">client</span><span class="p">.</span><span class="nf">GetQueueUrlAsync</span><span class="p">(</span><span class="s">"NAME_OF_DESTINATION_QUEUE"</span><span class="p">).</span><span class="n">Result</span><span class="p">.</span><span class="n">QueueUrl</span><span class="p">.</span><span class="nf">Dump</span><span class="p">();</span>
<span class="kt">var</span> <span class="n">rmr</span> <span class="p">=</span> <span class="k">new</span> <span class="n">ReceiveMessageRequest</span> <span class="p">{</span> <span class="n">QueueUrl</span> <span class="p">=</span> <span class="n">sourceQueueUrl</span><span class="p">,</span> <span class="n">MaxNumberOfMessages</span> <span class="p">=</span> <span class="m">10</span><span class="p">,</span> <span class="n">VisibilityTimeout</span> <span class="p">=</span> <span class="m">900</span><span class="p">,</span> <span class="n">WaitTimeSeconds</span> <span class="p">=</span> <span class="m">10</span> <span class="p">};</span>
<span class="kt">var</span> <span class="n">messages</span> <span class="p">=</span> <span class="n">client</span><span class="p">.</span><span class="nf">ReceiveMessageAsync</span><span class="p">(</span><span class="n">rmr</span><span class="p">).</span><span class="n">Result</span><span class="p">;</span>
<span class="k">do</span>
<span class="p">{</span>
<span class="s">$"Found </span><span class="p">{</span><span class="n">messages</span><span class="p">.</span><span class="n">Messages</span><span class="p">.</span><span class="n">Count</span><span class="p">}</span><span class="s"> messages"</span><span class="p">.</span><span class="nf">Dump</span><span class="p">();</span>
<span class="k">foreach</span> <span class="p">(</span><span class="kt">var</span> <span class="n">msg</span> <span class="k">in</span> <span class="n">messages</span><span class="p">.</span><span class="n">Messages</span><span class="p">.</span><span class="nf">Where</span><span class="p">(</span><span class="n">x</span> <span class="p">=></span> <span class="p">!</span><span class="n">seenIds</span><span class="p">.</span><span class="nf">Contains</span><span class="p">(</span><span class="n">x</span><span class="p">.</span><span class="n">MessageId</span><span class="p">)))</span>
<span class="p">{</span>
<span class="nf">SendToDestinationQueue</span><span class="p">(</span><span class="n">client</span><span class="p">,</span> <span class="n">destinationQueueUrl</span><span class="p">,</span> <span class="n">msg</span><span class="p">.</span><span class="n">Body</span><span class="p">);</span>
<span class="n">seenIds</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span><span class="n">msg</span><span class="p">.</span><span class="n">MessageId</span><span class="p">);</span>
<span class="n">sidContainer</span><span class="p">.</span><span class="n">Content</span> <span class="p">=</span> <span class="s">$"Seen Ids: </span><span class="p">{</span><span class="n">seenIds</span><span class="p">.</span><span class="n">Count</span><span class="p">}</span><span class="s">"</span><span class="p">;</span>
<span class="p">}</span>
<span class="n">messages</span> <span class="p">=</span> <span class="n">client</span><span class="p">.</span><span class="nf">ReceiveMessageAsync</span><span class="p">(</span><span class="n">rmr</span><span class="p">).</span><span class="n">Result</span><span class="p">;</span>
<span class="p">}</span> <span class="k">while</span> <span class="p">(</span><span class="n">messages</span><span class="p">.</span><span class="n">Messages</span><span class="p">.</span><span class="n">Count</span> <span class="p">></span> <span class="m">0</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">void</span> <span class="nf">SendToDestinationQueue</span><span class="p">(</span><span class="n">AmazonSQSClient</span> <span class="n">client</span><span class="p">,</span> <span class="kt">string</span> <span class="n">queueUrl</span><span class="p">,</span> <span class="kt">string</span> <span class="n">body</span><span class="p">)</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">response</span> <span class="p">=</span> <span class="n">client</span><span class="p">.</span><span class="nf">SendMessageAsync</span><span class="p">(</span><span class="n">queueUrl</span><span class="p">,</span> <span class="n">body</span><span class="p">).</span><span class="n">Result</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This just gets the urls for both queues and downloads messages from the source queue in batches of 10 until no more are returned.
For each batch returned, it creates a new message from that message body and sends it to the destination queue.</p>
<p>The seenIds variable just stores all previously seen message ids so if the visibility time expires of any messages that have already been processed while the script is still running, those won’t be resubmitted to the destination queue.</p>
<p>Note that in the above I am using all of the Async methods synchronously. For some reason, Linqpad was not executing properly with async on the day I was going this so I just made it synchronous. It seems to be working fine now but I just haven’t gotten around to <code class="language-plaintext highlighter-rouge">re-async-ifying</code> the script. This is just a one-off utility script, please don’t do this in Prod!</p>
<p>If you want to use this yourself, you’ll want to change the following placeholders in the script:</p>
<ul>
<li><strong>AWS_REGION_URL</strong> - The url for the AWS region you are using.</li>
<li><strong>AWSKEY</strong> - Your key for AWS.</li>
<li><strong>AWSSECRET</strong> - Your secret for AWS.</li>
<li><strong>NAME_OF_SOURCE_QUEUE</strong> - The name of the queue as viewed in the AWS console (i.e. not the full url).</li>
<li><strong>NAME_OF_DESTINATION_QUEUE</strong> - The name of the queue as viewed in the AWS console (i.e. not the full url).</li>
</ul>Alan ParrThis week I had the need to copy a load of messages from one SQS queue to another, below is the body of the Linqpad script I used to do it.