Forem: Alexander

YAML developer is a profession

Alexander — Tue, 28 Nov 2023 22:00:06 +0000

I decided to install k3d cluster locally to refresh my memories, and at the same time I found a reason to whine in this post. Not that I decided to move to DevOps/AWS for a reason https://dev.to/inv2004/it-trends-and-rates-20dn, but, the last thing I installed entirely myself in the prod of the orchestrator was docker swarm.

In general, if you do not go very much into the deeps, then k8s is arranged quite simply, and even logically. At some point you say aha, there should be a global k8s config repository somewhere, but then the second bottom opens, namely Helm, in which it is stated that everything that it does k8s, except for basic functions, it does poorly, we will help you with this. Naturally, the most difficult path possible is chosen. One of the main purposes Helm is the templating k8s of configs. It is used for this go template purpose, which immediately throws out a hodgepodge of embedded tags on you, so that you involuntarily start remembering the times when you programmed in JSP(or PHP if someone is closer). Moreover, even this is not enough, Helm can reuse your own components (rather meta-components), as a result, in order to understand how what is generated there, you need to dive not just into the template, but into someone's component and understand how it generates you an essentially simple yaml config that is waiting k8s. Helm v2 so in general, I put my own service on each node, but this was refused

I wouldn't have written this if it didn't explain to me why large companies have entire departments of so-called yaml developers doing this, a whole new profession in fact. Instead of tweaking something that doesn't look too bad anyway, namely k8s configs, a new abstraction is invented from above, another abstraction of components is added to this abstraction, and as a result, it's not surprising that there are people who support this five days a week.

As deja vu, I recall the aerobatics of the old days - generating the sendmail config via m4. The best people in the linux community have broken down on this, and this is probably one of the reasons why postfix replaced sendmail. I don't remember exactly, but a few lines (out of hundreds) looked like this:

define(`SMART_HOST', `relay:uucp-gateway')
LOCAL_NET_CONFIG
R $* < @ $+ .$m. > $* $#smtp $@ $2.$m $: $1 < @ $2.$m > $3

Check you CPU performance here

Alexander — Fri, 03 Nov 2023 19:23:00 +0000

There are a lot of different perf checks for CPUs, but, unfortunately, most of the them are more synthetic and do not reflect the tasks we do daily like devs. it is not surprise that devs' work is compilation

Here is benchmark I created which bootstrap Nim's language compiler

There are a lot of CPU's tested already, but you can add more if you want: https://github.com/inv2004/bench_nim_build

All you need is git+gcc or clang.

And do not forget to pug power cord if laptop!

Unlock AVX-512 on i5-12600

Alexander — Sun, 22 Oct 2023 21:16:37 +0000

It is possible to unlock AVX-512 on B660M bios.

At first I climbed to the forum where the bios is picking, I found that someone found something, and, by its naivety, I was going to setup BIOS with the AVX2 turned on and turned off, then compare, find the right byte and change ... I, of course , I didn’t find it, but I found the one who sent me a non -public version of F4 - thanks, - a link in the comment. They fixed the problems of productivity Non -K, but did not turn off the AVX512 - probably because it was removed from the site. Nevertheless, it was updated - and AVX -512 returned already in normal form. With a stress test from the CPU-Z, the frequencies that were scared before, did not notice.

It took time to rewrite the histogram from AVX2 on AVX512, and, I understood the complaints about the difficulty- part of the instructions that were on AVX2 could not be found, and I had to rewrite it differently.

In my case, AVX512 gave an increase of 25%. It was 8.21MS, it became 6.15MS. Not much, but not a few.

Conclusion: CPU 1/2 step closer to the GPU in vectorization, you can use it, and why it was blocked is not very clear.

Clean code

Alexander — Tue, 03 Oct 2023 21:22:00 +0000

After digging through the new code (golang, if that's important), I discovered a large number of interfaces that could easily be removed - they did not affect anything. But I decided to bring up this topic.

It became clear in advance that this was about the near-SOLID concept of “clean code”. I am neither against nor for, it has both pros and cons, but, in some aspects, in my direction of decision:

All articles on this topic begin with the same example: creating an access interface to a database (template repository) so that it can be implemented for several databases. This is motivated as follows: suddenly you intend to move from postgres to Mongo. A similar example, of course, can be expected in reality, but, as for me, it is still quite loose. I don't believe in simply moving from sql to nosql and back for any project, more than todo. Motivation number two is to implement it for both postgres and sqlite, because it is difficult for a programmer to run postgres locally. From this moment you already want to start giggling, having provided for a certain number of additional problems, you choose a communication method, where the foundation of the project changes radically during development and operation. I would like to add that, for sure, there is already an abstraction for the sql database, where to replace the database you only need to replace the driver.

In my head, it so happens that interfaces are created at the moment when two entities make up one contract, but all attempts to reach two entities in the examples above look strange. But, the trick here is to take off his hat, and from the hat the metal unit tests are being mocked, namely the unit. Do you have some code, but can't come up with a second entity? - then this entity will be tests for this code, any code has (should have) tests, therefore, there is a second entity, therefore, everything should be as abstract as possible.

And this logical twist, because of the example, shows better, it seems strange to me, i.e. tests pull the code onto themselves, and not vice versa. Again, I don’t want to say that this shouldn’t be said - everyone decides for themselves whether TDD or not TDD, but presenting this as the only correct option for everything seems strange to me. I add that if you replace unit tests with integration tests, which I have seen quite often, then this concept falls apart again.

It would be possible to add about poly- and monomorphism - generics and duck typing, but there’s still a lot

added - A good example of the convenience of this concept that was given to me is caching

When 20.5 billions is not enough

Alexander — Wed, 27 Sep 2023 18:57:51 +0000

Trying to login with my old account:

Trying to register the account:

IT trends and rates

Alexander — Tue, 26 Sep 2023 22:41:51 +0000

Prologue

Recently, I had a reason to revisit freelance exchange. About 5 years ago I did it for the first time because I was so tired of enterprise and Java and I was trying to make a decision about whether to stick with my boring, but well-paying technologies, or move to more interesting but less payed fields.

Dataset

For those readers who aren't interested in the technical process, you can skip to the Conclusion section

I explored the IT category and found that it contains approximately 5000 offers. There were about 100 pages with 50 items per page, I went through the hundred pages and saved a .har file in chrome, split it by page request and saved them into separate files. Here comes the data for analysis.

You can find the most trending tags based on these offers in the picture:

As we can see, DevOps is the trending topic at the moment. Pretty funny, considering that just a few years ago, when I partly joined a DevOps team at one of the largest banks in Europe, my manager could not define what "DevOps" meant. But now, it is a top trend.

Rates per hour seemed noticeably lower than I expected. After a little investigation, I found a lot of offers with significantly lower rates than usual. I can still remember certain regions offering lower rates in the past. Typically, you want to target regions where rates are equal to or higher than those in your local region. Therefore, I decided to filter out those offers from regions with reduced rates, based on my location.

The final rating is as follows:

It is just top of the table. Here’s some explanation:

Min and Max represent the are averages (to smooth it a bit and to avoid spikes) of minimum and maximum rates respectively (where the rate is specified), Avg is an average of the Min and Max. Project price is an average of fixed project prices, which is rather insignificant due to it’s unrelatedness to work duration.

Relations

I thought it would be interesting to find correlation what exactly what some tags mean, so I did a simple thing: I linked tags from the same offer, where the central force of a node is the number of its links. The greater challenge was finding a good online tool for graph visualization.

One of them was cosmograph.app, which suggested our IT industry is alive and rotates around SQL

So I generated the graph in graphviz. Too many minor links resulted in a somewhat cluttered distribution, so I decided to limit each node to those with 75 or more links

To explore it in better detail, you can find it here https://t.ly/1Y3ur

From the graph, it appears that these days, the term DevOps mostly means AWS, then Docker and CI/CD. Python also seems to be the main language for DevOps also

There are two clusters here, the second one is about Security, I am not sure I know the major diff between information/network and other security issues. Also, it is necessary to note that Upwork contains a lot of “security” stuff like “hack my ex’s instagram”

I wanted to detect more clusters, but, as I mentioned earlier, I struggled to find good tools that could do it quickly. Also, it was pretty clear that it is possible to merge some nodes like “AWS” and “Amazon EC2”, but did not want to spend too much time on it

Conclusion

Finally, I created a table focused on the topics which are most interesting to me, specifically programming languages:

From the table and the graph it is quite clear that DevOps goes hand to hand with AWS, sometimes it goes with Azure and it tends to pay a bit better. Knowledge of Docker is must-have If you want to work in DevOps. After that, K8s and TerraForm could also extend your feeding ground. “DevOps” contains “Dev” — typically involves Python, knowing Ansible would also be benefit.

The topic of programming languages is of particular interest to me:

Python is language for everyone, for tooling and not only. JavaScript also ranks high, but I am surprised with the rate I see. Next surprise for me is unreasonable (sorry) PHP with reasonable price for it. Java experienced some stagnation about 10 years ago from my perspective, but it remains popular. C# is half as popular as Java. C++ is still performing well as an older technology. Typescript, I believe, should be merged with JS. Flutter does not look very popular, but the number of offers involving it is quite low, which might have resulted in it's low rate. The same applies to C, but I had expected C to rank higher.

When I decided to quit Java I bet on Golang - mature enough for prod work and was increasing in popularity, but I have a feeling that it is ~ on the same level it was before. Maybe, if I were brave enough, I would invest into AWS when I made the decision. At least that seems like a reasonable move based on the current market.

Rust is mostly supported by crypto, but its presence on the market is a good result. Kotlin ranks slightly higher than Rust, and surprisingly, I found that it is more often used in backend development than in Android development, as it was originally positioned.

Databases. MySQL tops the list, which is expected — the number one choice for those who prefer not to deliberate over their database selection. Postgres is the next one, followed by Mongo and Redis. Unfortunately, ClickHouse is nearly invisible, but I continue to follow interesting projects that involve it.

Epilogue

This analysis is based on freelance exchange, but the results can still be applicable to permanent employment as well, with some deviation for expensive regions. Of course, there are some banks which are still using rare and aged technologies like APL (❤️) or Fortran, and it's quite common to stay in one department for many years. Such scenarios could be interesting and even profitable, but they may not always offer good job security perspective. If you're in the process of making a career decision, this article might help you to make a decision based on different criteria: to find the most profitable tech or tech which you could improve your skills. It also allows you to compare your income against the minimum to maximum rate scale based on your proficiency level, and find out if you're being paid reasonably or not. Moreover, this analysis could also provide insights into the idea of freelance, which can be profitable, but could be more risky too.

PS

Some data that is not only about IT: