Forem: Maiko Miyazaki

How to create a related post component with gatsby-remark-related-posts plugin

Maiko Miyazaki — Tue, 22 Jun 2021 14:39:02 +0000

This article is originally published on miyazakimaiko.com. Photo by Girl with red hat on Unsplash.

Motivation

I recently deployed my personal blog miyazakimaiko.com, but I had difficult time dealing with a related post component when I build this website. I found a couple of ways to implement it without a plugin, but I wanted it as simple as possible, preferably complete it on the build time, and didn't mind using a plugin for it. Then I found gatsby-remark-related-posts plugin, which seemed feasible, but I couldn't find solutions to some of the issues I came across when implementing it. So this article is for the people who stuck like me. I'll summarise implementation steps to make it as easy as possible to understand.

Requirements

Using allMarkdownRemark for querying .md files
Unfortunately, it seems like it's not compatible with .mdx files or mdx query. If you find it possible, please let me know in the comment section!

gatsby-remark-related-posts plugin is installed

To install the plugin, please refer to this page.

File hierarchy

project
│   gatsby-config.js
│   gatsby-node.js   
│
└───content
│   └───blog 
│       └───article1   
│       │   │   index.md
│       │   │   featured_image.jpg
│       │
│       └───article2
│       │   │   index.md
│       │   │   featured_image.jpg
│       
└───src
│   └───templates 
│       └───article.js
...     ...

This folder is used as the example.

Implementation

gatsby-config.js

Set directory where you include your markdown files in posts_dir, like this:

plugins [
  {
    resolve: "gatsby-remark-related-posts",
    options: {
      posts_dir: `${__dirname}/content/blog`,
      doc_lang: "en",
    },
  },
]

gatsby-node.js

Query part
This code below allows the plugin to create relatedFileAbsolutePaths field in the each MarkdownRemark node. Details: https://www.gatsbyjs.com/plugins/gatsby-remark-related-posts/

createPages action part
Pass the relatedFileAbsolutePaths as a variable to the individual page. To do so, we can simply set relatedFilePaths: post.node.fields.relatedFileAbsolutePaths.slice(0,4) inside context. You could just pass whole list of related file absolute paths, or specify how many article you want to pass by slicing the list.

exports.createPages = async ({ graphql, actions }) => {

  const { createPage } = actions

  const blogPostCollection = await graphql(
    `
      {
        allMarkdownRemark {
          edges {
            node {
              fileAbsolutePath
              fields {
                relatedFileAbsolutePaths
              }
              frontmatter {
                title
                slug
              }
            }
          }
        }
      }
    `
  ).then(result => {
    if (result.errors) {
      throw result.errors
    }

    // Create an individual article page for each article.
    const posts = result.data.allMarkdownRemark.edges

    posts.forEach((post, index) => {
      createPage({
        path: `blog/${post.node.frontmatter.slug}`,
        component: path.resolve(`./src/templates/article.js`),
        context: {
          slug: post.node.frontmatter.slug,
          relatedFilePaths: post.node.fields.relatedFileAbsolutePaths.slice(0,4),
        },
      })
    })

    // Create blog list pages here..

    return null
  })

  await Promise.all([blogPostCollection])
}

Caution with the Query: In my blog, I set slug as a part of frontmatter, but most of cases you have slug field.

templates/article.js

We can simply query related posts when fetching the single article like below. Make sure to mention the argument $relatedFilePaths: [String], which we passed to this page in the gatsby-node.js file.

export const query = graphql`
    query BlogTemplate($slug: String! $relatedFilePaths: [String]) {
        markdownRemark(frontmatter: { slug: { eq: $slug } }) {
            ...
        }
        allMarkdownRemark(filter: {fileAbsolutePath: {in: $relatedFilePaths}}, limit: 4) {
            edges {
              node {
                ...
            }
        }
    }

In the template, you'll be able to access the main article data from props.data.markdownRemark and related posts data from props.data.allMarkdownRemark.edges.

Hooray! Now I have a related posts component available to the each page 🎉

Consideration

This plugin accumulates related posts on the build time rather than runtime, so I'd say it's highly efficient. However, I do not have much articles on my blog so I don't yet know tell how it affects the speed of the build time. I'll do more research about it, and update this article when I find any realization in the future.

Thanks for reading. If you have any opinion or question, please leave a comment below!

20 things I learned about Scrum from Learning Agile

Maiko Miyazaki — Sun, 02 May 2021 15:55:26 +0000

As I will be joining a company where Scrum seems prevalent, I decided to read a book called Learning Agile written by Andrew Stellman and Jennifer Greene. It's such a great book that given me a better idea of what it's like to be a part of a scrum team.

In this article, I'll summarise 20 things from the book that I found particularly useful and important for a complete newbie including myself.

1. Each role has a different perspective on user stories

There are many roles to form a Scrum team such as Product Owner, Scrum Master, Team lead, Lead developer, etc.

For developers, for instance, a user story is the functionality written in a simple and easy way to understand. We use user stories to break up and organise tasks. For the team lead, user stories are a tool to communicate with developers to help them figure out what should be done next.

The product owner, on the other hand, sees the list of user stories as values that will be delivered to the company. User stories are the tool to communicate with customers to validate what they are looking for.

2. Before diving into coding, have discussions to see user stories in the shoes of other roles

Because understanding of a user story can differ depending on the roles, developers could make wrong assumptions on what they are making. Without taking the time to understand user stories correctly, we could end up making big changes to the codes that could have been avoided.

3. It's not enough to have the "correct" process or the "best" practice

Take the time and have conversations with your team to understand why you are doing what you are doing with process and tools, and how people on the team work together with it. Things can go wrong if we just follow them blindly.

4. Focus on working software rather than comprehensive documentation

There are many kinds of documentation are very useful for the team including wireframes or sequence diagram, but is not the goal to the project. Documentation is only a means toward the end of working software.

5. The term "Working software" has a specific definition

"Working software" is not just the software that is functioning. In a Scrum team, Working software is the software that adds value to the organization.

6. Be cautious when you are constantly "negotiating" to protect yourself

When teams don't collaborate toward the single goal of delivering working software, they often treat each other as if they're working on a contract. It may protect us from getting into trouble when things go wrong, but is highly counterproductive.

7. Constantly look for changes

If we work the wrong plan, we'll build the wrong product. However, it's common for a person who built a plan to resist changing them. Even that's the case, changes cost more later on, so constantly make effort to find out necessary changes as early as possible.

8. Principles add life, character and heart into Practices

A team cannot get the full benefit of solid practices without principles.
In order to gain the full benefit of agile practices, do not kill collaboration by only thinking about developer stuff. Have a lot of communication to see the project in the perspective of a team, not your own perspective as a developer.

9. Scrum Master doesn't own the plan

Everyone on a scrum team owns the project. The Scrum Master guides the team's decisions, use of Scrum and its principles.

10. Product Owner makes decisions on behalf of the business

Product Owner doesn't just sit around to wait for the sprint to finish. They work with all of the team members every day, being the voice of the business, to help the team to understand the value of software. It is important to actually listen to the product owner's opinions and ideas.

11. Programming isn't the only important thing in a project

Many programmers take the attitude, and many companies encourage this when they structure their teams around a core set of technical people. Be wary when you see everyone else except developers as "outsiders".

12. Be cautious when prioritising your own professional goal

When we have this mindset, we are not committed but merely involved in the project. During a sprint, the success of the project is more important than getting closer to your own professional goal. For instance, if we choose technology for the project because we want to learn it, we are putting weight on our own professional goals, rather than the success of the project.

13. The more people outside the team involved, the more value he or she can potentially add

For instance, users of the software should be involved because their opinion matters. That's one of the reasons to release working software to the users on a regular basis to keep them involved.

14. Collective commitment rather than committing to individual tasks

When a team makes a collective commitment, that means each team member agrees and commits genuinely to deliver the most valuable software they can to their users and stakeholders at the end of each sprint.

15. Task assignments come from the team member themselves

The team members themselves are the source of the task assignments.

So ask yourself what you think you should do, rather than simply asking the team lead what your next task is.

16. Daily Scrum is not a way for the team lead to manage the schedule

Daily Scrum is a stand-up meeting. It's for the team to plan the next day's work but not for the team lead to get status updates. To achieve it, each team members inspect the work that the team is doing. It gives the flexibility to make the right decisions including task assignments in the last responsible moment.

17. Ask each other 3 questions every day on the Daily Scrum

Ask these three questions:

What have I done since our last meeting?
What am I planning on doing between now and our next meeting?
What roadblocks are in my way?

By everyone on the team inspecting what everyone else is doing in the same way every day, a lot of problems get caught before they become costly.

18. Pleasant surprises for users/stakeholders can be almost as damaging as unpleasant ones

Avoid any surprises at all cost. To do so, keep everyone up to date on all of the changes they discover during the daily scrum. That's why the attendance of users/stakeholders to the Daily Scrum is valuable.

19. Have a concrete definition of "Done"

For developers, it's easy to feel like it's finished when looking back at what they've just built. But calling it "Done" in this situation can create problematic misunderstanding on the team because your "Done" is ambiguous for anyone else on the team.

Get used to refer Condition of Satisfaction/Acceptance Criteria which defines the concrete definition of "Done" for the task. For each work in the task to get it done, it must also be understood, built, tested and deployed.

20. A lifetime to master

The basic practices of Scrum are straightforward to learn and adapt, but there'll be many challenges if we want to master it.

While I was reading through the book Learning Agile to write this summary, I was also reading a book called "Courage to be disliked" written based on Alfred Adler's theories in tandem. As reading through this book, I noticed that there are many things in common with Scrum and Alfred Adler's theories. It was an interesting realisation.

If you don't buy some practices of Scrum, I highly recommend reading the book Courage to be Disliked, and not to mention, Learning Agile.

Thanks for reading. If you have any opinion or question, please leave a comment below!

C# date & time conversion CheatSheet

Maiko Miyazaki — Wed, 28 Apr 2021 14:59:14 +0000

When dealing with data, we often find ourselves handling date and time. Although it is not particularly difficult, I often find myself googling the same conversion over and over again.

This post will share a cheat sheet of date and time conversion to eliminate such little time-consuming procedures and explain each technique in the following sections.

Open in PDF

1️⃣ String to DateTime object
2️⃣ String that is not parsable with Parse() to DateTime object
3️⃣ DateTime Object to String
4️⃣ String to String
5️⃣ DateTime Object to Unix Timestamp in Seconds
6️⃣ Unix Timestamp in Seconds to DateTime Object
7️⃣ UTC timezone String to DateTimeOffset object in a different timezone
8️⃣ 24-hour time String to 12-hour time String
9️⃣ Download PDF file

String to DateTime object

Input Example: String "31-12-2021"
Output: Datetime object "31/12/2021 00:00:00"

We can use DateTime.Parse() to create a DateTime object from a string. Parse method takes target string as an argument.

String that is not parsable with Parse() to DateTime object

Input Example: String "31122021"
Output: Datetime object "31/12/2021 00:00:00"

We can use DateTime.ParseExact() to create a DateTime object from a string that is not parsable with Parse() method. ParseExact() takes three arguments -- target string, a string to explain the format, and Format Provider which pecifies format of the DateTime conventions associated with language, region, calendar, etc.

DateTime object to String

Input Example: DateTime object "31/12/2021 00:00:00"
Output: String "2021-12-31"

We can use ToString() methodto convert a DateTime object into String. The method takes desired string format as an argument.

String to String

Input Example: String "31122021"
Output: String "2021-12-31"

To convert a string into a different format of a string, we first need to convert it into a DateTime object. Then we can use ToString() methodto specify a format and convert it into a string.

DateTime Object to Unix Timestamp in Seconds

Input Example: DateTime Object "31/12/2021 00:00:00"
Output: Long 1640908800 (Unix Timestamp in seconds)

To convert DateTime Object into Unix Timestamp in seconds, we first need to convert the DateTime into a DateTimeOffset object. Then we can use ToUnixTimeSeconds() methodto convert it into a timestamp in Long.

Unix Timestamp in Seconds to DateTime Object

Input Example: Long 1640908800 (Unix Timestamp in seconds)
Output: DateTime object with "31/12/2021 00:00:00"

To convert a Unix timestamp (in seconds) into a DateTime object, we first need to convert the timestamp into a DateTimeOffset object. Then we can use DateTime propertyto get the DateTime object.

UTC timezone String to DateTimeOffset object in a different timezone

Input Example: String "31-12-2021 08:33:00 -0400" (New York timezone)
Output: DateTime object "31/12/2021 21:33:00 +09:00" (Tokyo timezone)

There are two main steps to convert a UTC timezone String into a DateTimeOffset object in a different timezone.

Firstly, we need to calculate the UTC offset of the output timezone. To do so, we can use TimeSoneInfo.FindSystemTimeZoneByIdto get the TimeZoneInfo object, and then calculates the offset using GetUtcOffset() method.

Next, we create a DateTimeOffset object from the input string using DateTimeOffset.ParseExact(). After that, convert the DateTimeOffset using ToOffset() method with the offset calculated in the first step.

24-hour time String to 12-hour time String

Input Example: String "20:00"
Output: String "08:00 pm"

We first need to convert String into a DateTime object. Right after, 12-hour time string can be created using ToString method.

Download PDF file

Open PDF

Thanks for reading. If you have any opinion or question, please leave a comment below! In the meantime, please follow me on Linkedin @Maiko Miyazaki or Twitter @Maiko Miyazaki

Resources

Converting between DateTime and offset - Microsoft docs
DateTimeOffset.ToOffset - Microsoft docs
List of timezone IDs for use with findtimezonebyid - Stackoverflow

Python date & time conversion CheatSheet

Maiko Miyazaki — Sat, 10 Apr 2021 10:17:59 +0000

When dealing with data, we often find ourselves handling date and time. Although it is not particularly difficult, I often find myself googling the same conversion over and over again.

In this post, I'll share a cheat sheet of date and time conversion to eliminate such little time-consuming procedures and explain each technique in the following sections.

Open in PDF

1️⃣ String to DateTime object
2️⃣ DateTime object to String
3️⃣ String to String
4️⃣ UTC timestamp in seconds to DateTime object
5️⃣ UTC timestamp in seconds to String
6️⃣ UTC time string to UTC DateTime object in different GMT
7️⃣24-hour time String to 12-hour time String
7️⃣Download PDF file

String to DateTime object

Input Example: String "08/04/2021"
Output: Datetime object "2021-04-08 00:00:00"

We can use datetime.strptime to create a DateTime object from a string. strptime method takes two arguments -- target string and string to explain the format of the target string.

DateTime object to String

Input Example: DateTime object "2021-04-08 00:00:00"
Output: String "08/04/2021"

We can use datetime.strftimeto convert a DateTime object into String. This strftime method takes two arguments -- DateTime object and String to specify desired string format.

String to String

Input Example: String "08/04/2021"
Output: String "2021-04-08"

To convert datetime string into a different format of a string, we first need to convert it into a DateTime object. Then we can use strptimeto specify and convert into the desired format.

UTC timestamp in seconds to DateTime object

Input Example: float 1617836400.0 (UTC timestamp in seconds)
Output: DateTime object with "2021-04-08 00:00:00"

To convert float of timestamp into a DateTime, can use datetime.fromtimestamp(). It takes timestamp as an argument and returns DateTime object.

UTC timestamp in seconds to String

Input Example: float 1617836400.0 (UTC timestamp in seconds)
Output: String "08/04/2021"

To convert float of timestamp into String, we first get a DateTime object. Using strftime method, we can then create desired format of String.

UTC time string to UTC DateTime object in different GMT

Input Example: String "08-04-2021 08:33:00-0400" (New York timezone)
Output: DateTime object "2021-04-08 21:33:00+09:00" (Tokyo timezone)

To convert an UTC time string, we first convert it into a DateTime object. After that, we can use astimezone method to create a DateTime object in specified GMT.

24-hour time String to 12-hour time String

Input Example: String "20:00"
Output: String "08:00 PM"

We first need to convert String into a DateTime object. Right after, 12-hour time string can be created using strftime method.

Download PDF file

Open PDF

Thanks for reading. If you have any opinion or question, please leave a comment below! In the meantime, please follow me on Linkedin @Maiko Miyazaki or Twitter @Maiko Miyazaki

Let's remember the concept of Abstraction, Generalisation, Decomposition and Encapsulation in one go!

Maiko Miyazaki — Fri, 26 Mar 2021 16:42:48 +0000

Did you just begin to learn Object-Oriented Design/Programming recently? Are you struggling to grasp the general idea of Abstraction, Generalisation, Decomposition and Encapsulation? Are you not sure about the differences between each concept?

If that sounds like you, you are not alone! For newbies, there are so many concepts to remember and it can be overwhelming.

In this post, we'll be looking at the summary of each principle and then move onto each of them. To help us remember the concepts in one go, we will be looking at the four main principles in a setting simulating a retail company similar to IKEA.

As a result, we'll be able to grasp the general ideas and understand why they are useful.

Summary

Principle	Goal	Related words	Examples
Abstraction	Reduces complexity by showing only relevant information	abstract class, abstract method, extend	A row header for chairs
Generalization	Reduces complexity by having an umbrella term	superclass, subclass, interface, implements, extends	Furniture for chairs, tables and chests
Decomposition	Reduces complexity by separating problems	association, aggregation, composition	Fitted Kitchen
Encapsulation	Restrict access to data to protect them and reduce complexity	getter method, setter method, private, protected, public	System of a restaurant section

Now let's dive into details.

Abstraction

Abstraction can reduce complexity by hiding irrelevant information of things. In other words, it clarfy things by showing only relevant information.

Let's look at examples.

Imagine we sell chairs in our retail company. On our website, we listed our products and information to help customers to make purchase decisions. The product pages of chairs would contain those information listed below without having concepts of abstraction:

What do you think about the representation? Are some of them irrelevant to customers to make purchase decisions?

In such a situation, we usually show only relevant information to help make our customer's purchase decisions without confusion.

What we can see here is a much less complex structure because irrelevant information were ignored and rest of them are organized.

Abstraction is like creating the row header in the picture. They can become attributes within a Chair class. As a result, it is much easier to create a new type of chair because you have column headers, which is a class in OOP, as a template.

Generalization

Generalization can reduce complexity by creating an umbrella term. In other words, having an umbrella term can reduce redundancy and improve efficiency.

Now, let us continue to use the example of the retail company to look at how generalization can be used.

At the moment, we sell chairs, tables and chests in the retail section of our company. At the moment, we have a column header for chairs including Product name, Type, Article number, Width, Depth, Height, Seat width, Seat depth, Seat height, Max. seat height, Tested for.

When we looked at each title carefully, you may notice that some of them are obviously necessary for tables and wardrobes as well, but others are only applicable for chairs.

For instance, it is clear that Width, Depth, Height must be included in any products we have, because they are the key factors for any furniture to evaluate if it is compatible with the customer's usage.

If that's the case, we can state that Width, Depth, Height as well as Product name and Type are must-have information to include in an umbrella term which we could name it Furniture.

Decomposition

Decomposition can reduce the complexity of a problem by separating it into smaller chunks.

Imagine we started selling more products including kitchen appliances in our retail company. Now, we are planning to sell fitted kitchens and set the fitted kitchen samples at our shop.

So far, we have specified what to include in a fitted kitchen sample without decomposition.

What do you think of it? Isn't it complicated and unclear? For now, the list is small enough for the sake of simplicity, but what could happen if we include all the functionalities?

Without decomposition, an object could have too many responsibilities and purpose.

In such a situation, we usually separate the responsibilities into smaller chunks to make things clearer and easier to maintain.

There are three types of relationships between the decomposed Fitted Kitchen object and its components, namely Composition, Aggregation and Association.

Composition (with Dimension)

Composition creates the strongest relationship. In the example above, the Fitted Kitchen could have a relationship with Dimension because when the Fitted Kitchen is destroyed, Dimension should also be destroyed. In other words, Dimension cannot exist without the Fitted Kitchen itself.

Aggregation (with Tap, Ventilation fan, Oven...)

Aggregation is a looser relationship than composition. In the example, Tap, Ventilation fan, Oven, Hob, Fridge, Freezer and Cupboard could have aggregation relationships with Fitted Kitchen if we decide that Fitted Kitchen should not be presented to customers without one of the components. That being said, each component can exist or be sold individually even though the Fitted Kitchen is destroyed.

Association (with Designer)

Association is a loose relationship. In the example, Designer could have an association relationship with the Fitted Kitchen, if the existence of Designer does not affect the existence of the Fitted Kitchen and vice versa. In other words, the Designer can exist and do their job even though the Fitted Kitchen is destroyed.

Encapsulation

Encapsulation can reduce the complexity of issues by bundling data and the methods that work on the data within one unit.

Let's say we are planning to have a restaurant section in our retail company like IKEA does.

We decided to have three menus like below:

Dish Name	Comes with	Price
Grilled Salmon	Small soup, French fries	8.50
Vegetable Soup	Small Salad, A slice of soda bread	8.00
Beef Sandwich	Small Salad, Crisps	8.50

Without encapsulation, we are forced to have a buffet-style bar that customer can serve themselves freely. To put it another way, a customer is able to change the Small Salad to Grilled Salmon as the side, because they are set to public. As a result, the price and the system of the restaurant would break.

Encapsulation enables the restaurant system to have staff who control the changes in the dishes. In other words, each component of the dish is set to protected or private. They can allow a customer's request if it's allowed, or reject if they cannot change so that prevent direct changes and mistakes.

Conclusion

As I found these design principles terms a bit demotivating to remember at first, I prefer to include all of them in a single setting. I hope those examples help you to remember and distinguish the differences!

Thanks for reading. If you have any opinion or question, please leave a comment below! In the meantime, please follow me on Linkedin @Maiko Miyazaki or Twitter @Maiko Miyazaki

References

Object-Oriented Design - Coursera
What is Abstraction in OOPs? Java Abstract Class & Method - Guru99
OOP Concept for Beginners: What is Abstraction? - Stackify
What's the difference between abstraction and generalization? - Stackoverflow
Generalization - SAN JOSÉ STATE UNIVERSITY
Premature Generalization Is Evil - Wiki Wiki Web
Decomposition - BBC Bitesize
OOP Concept for Beginners: What is Encapsulation - Stackify

Newbie's Cheatsheet: Commonly used verbs for naming functions, methods and variables

Maiko Miyazaki — Mon, 15 Mar 2021 14:47:39 +0000

Do you follow your instinct when choosing a verb in your function?

Do you use a number of synonyms for the same purpose inconsistently?

If the answer is yes, you are not the only one.

In order to write a clean code, or at least to attempt it, we need to know what words are commonly used so that we can use them, and people can understand easier what we are trying to do.

At the same time, we need to stick with one word for one concept rather than using synonyms inconsistently.

In this post, I'll summarise commonly used verbs found in programming books and Github regardless of the type of programming language, so we can find the best-suited verbs for our own needs.

On the tables below, we'll see 5 rows:

The verb
The number of times the verb is written on Github to give us some ideas on how commonly it's used
Its general definition
Examples
Synonyms to avoid using to express the same concept

Just bear with me, this is not a complete list of common words, so please do let me know in the comment if you find some of the definition is too biased on certain programming languages. I'll also be grateful if you could teach me any other verbs that you think it's common!

I hope this post helps us, newbies choose a suitable verb and stick to it.

1️⃣ Alteration
2️⃣ Creation
3️⃣ Establishment
4️⃣ Obtainment
5️⃣ True or False Statement
6️⃣ Conclusion

1. Alteration

Verb	Written on Github	Definition	Examples	Synonyms to avoid
set	2B	Often used to put data in an existing resource such as an attribute of an object.	set_name(), set_store_id()
change	668M	Often used when a whole thing, such as image, is replaced by something else	change_image(), change_colour()	edit
edit	325M	Often used same as change. It could be used especially when action is responsible for rendering the view.	edit_title(), edit_record()	change
update	739M	Often used when one or more of the components is updated as a result, and something new could also be added.	update_file()
add	1B	Often used to add something into a group of the things.	add_item(), add_member()	append
append	287M	Often used same as add. It could be used when it does not modify the original group of things but produce the new group.	append_item(), append_element()	add
remove	824M	Often used when a given thing is removed from a group of the things.	remove_record(), remove_item()	delete
delete	455M	Often used same as remove, but it could also render nonrecoverable.	delete_record(), delete_item()	remove
save	478M	Often used when preserving data to avoid loss.	save_json(), save_order()	store
store	396M	Often used the same way as save.	store_json(), store_order()	save
disable	586M	Often used to configure a resource an unavailable or inactive state.	disable_user()	hide
hide	237M	Often has the same intention as disable, by hiding it.	hide_field()	disable
split	276M	Used when separating parts of a resource.	split_table(), split_group()	separate
separate	151M	Often used the same way as the verb split.	separate_table(), separate_group()	split
merge	312M	Often used when creating a single resource from multiple resource.	merge_records(), merge_tables()	join
join	220M	It can be used in a same way as merge.	join_records(), join_tables()	merge

2. Creation

Verb	Written on Github	Definition	Examples	Confusing synonyms
create	1B	Used when creating a resource.	create_package(), create_directory()	make, generate
make	797M	Often used in a same way as create.	make_package(), make_statement()	create, generate
generate	286M	Often used in a same way as create.	generate_directory(), generate_statement()	create, make
copy	1B	Used when creating a resource with the same structure and data as the original one.	copy_file(), copy_graph()	clone
clone	147M	Could be used in a same way as copy.	clone_file(), clone_graph()	copy

3. Establishment

Verb	Written on Github	Definition	Examples	Confusing synonyms
start	1B	Generally used when initiating an operation.	start_listening()	begin
begin	342M	Often used in a same way as start.	begin_listening()	start
open	854M	Used when changing state of a resource to make it accessible or usable.	open_file()

4. Obtainment

Verb	Written on Github	Definition	Examples	Confusing synonyms
get	2B	Generally used to obtain a resource.	get_data(), get_store_id()	fetch, retrieve
fetch	146M	Can be used in a same way as get.	fetch_data(), fetch_store_id()	get, retrieve
retrieve	116M	Can be used in a same way as get or fetch	retrieve_data(), retrieve_store_id()	get, fetch
read	1B	Used when acquiring data from a source.	read_file()
find	672M	Used when looking for an unknown data in a container.	find_element(), find_keyword()	search
search	438M	Generally used in a same way as find. It may refer to look for an unknown data from multiple containers.	search_element(), search_keyword()	find
close	492M	Used when changing state of a resource to make it inaccessible or unusable.	close_file()

5. True or False Statement

Verb	Written on Github	Definition	Examples
is	3B	Used when defining state of a resource.	Method: is_available(), is_correct() Variables: name_is_available, date_is_correct
has	1B	Used to define whether a resource contains a certain data.	Method: has_name(), has_date() Variables: person_has_name, course_has_date
can	2B	Used to define a certain ability of a resource.	can_load(), can_create()
should	1B	Used to define a certain obligation of a resource.	should_render(), should_print_message()

6. Conclusion

Writing clean code is not a skill that we can obtain overnight. But at the same time, I believe that we can improve the quality of code significantly and constantly by learning the wisdom of other people's clean code. To write this bost, I gained so many ideas from the book Clean Code written by Robert C. Martin and many codes on Github. I especially recommend the book for anyone who is just started coding. Many of the contents might seem going too far for newbies, but I am certain that this book will be my forever reference for my career as a developer.

Thanks to these people who share their own hard work!

Thanks for reading. If you have any opinion or question, please leave a comment below! In the meantime, please follow me on Linkedin @Maiko Miyazaki or Twitter @Maiko Miyazaki

Focus on being FAST at getting work done rather than Quality or Quantity.

Maiko Miyazaki — Sun, 07 Mar 2021 00:08:25 +0000

Have you ever been in a situation where you cannot tell which is more important -- Quality or Quantity? Or, have you felt that you never fulfil quality nor quantity on your tasks?

I have. In many situations, including but not limited to college tasks, freelance work, and blog posts. But I've never been happy with choosing one of the other simply because sometimes you need both quality and quantity.

So how can we overcome this contradiction?

As you can see in the title, focusing on speed rather than quality or quantity would be the solution.

In the former section of this article, you'll know why SPEED is more important than quality or quantity.
In the latter section, you'll see 3 concrete tips to speed up.

Why SPEED is the key?

If we have your own strategy to speed up your tasks, then we'll be able to change your focus on quality or quantity easily depending on the requirements.

Although we often hear the Quality vs Quantity debate, it's not that straight forward in a real-life. For instance, your team may tend to value quality over quantity, but one of your bosses might think highly of quantity rather than quality. The same is applicable to clients. We are required to be flexible.

Moreover, if you are a junior developer who does not have experience in the industry, you might find yourself lack of skills to fulfil neither quality nor quantity.

But what if we have our own strategy to complete tasks faster?

If we can focus on speed, that means we'll be able to complete a task faster than we used to be. After completion, we'll be able to ask what they are looking for further from us, either quality or quantity. Because you now have extra time, you'll be able to improve it further as required.

This is the main goodness of focusing on SPEED.

How to SPEED UP?

There are three easiest principles to improve the speed of your tasks.
Provide answer right away, Start right away and Put forward earlier.

We may think that it is not easy. But for us, these three ways are the easiest ones to achieve higher speed on tasks since it does not require skills but just mindset.

Moreover, they are the only ways to be faster than our counterparts who are more experienced than us. They can for sure complete tasks earlier than us if we start at the same time on a task.

Provide answer right away, Start right away and Put forward earlier. These principles can be the best way to achieve a higher speed on tasks.

1. Provide an answer right away

By answering right away, we'll be able to remove obstacles as early stage as possible.

In many situations, you might feel you need several hours to give some answer. But such a slight delay can get bigger. Even if that's not the case, these delays do not give a positive impression to the person who is waiting for your answer.

The good news is that it can be eliminated easily by answering right away and tell them what your concern exactly is.

For instance, imagine you are asked if you want to work on certain tasks or project. But you are not sure if you have the required skills for it.

Even though you may feel you need some time to give an answer, tell them right away some kind of answers. It could be letting them know the date you can give the final answer, or telling them that you want to do it but you are not sure with certain skills.

The point is to share your circumstances and ask for help as early as possible.

By doing so, your boss or client can provide you with support as early as possible and you don't need to waste your time considering on your own.

2. Start right away

If we start working on a task on the day we were asked to do it, we have a chance to be able to complete it faster than our experienced counterpart.

In other words, we will not be able to reach a higher standard in quality nor quantity if we don't start working on it earlier.

This might seem very simple and obvious, but many people, including me, tend to underestimate the power of starting early ending up procrastinate.

Start working on the task on the day you are asked to do it, regardless of the deadline. If we could manage to start working on the tasks earlier, then we can be faster. It's that simple.

3. Put forward earlier

By putting forward earlier, you'll have a chance to ask for feedback if you should brush up on quality or quantity by the actual deadline. This mindset is vital to have if we are newbies in the field.

Have you ever had this experience -- looking back at your own work that you have spent a long, long time to achieve higher quality, but when you look at it now, the quality is shite? Or you thought you did very well on assessment but it turns out not as good as you expected?

Because we don't have much experience, we oftentimes misevaluate the quality or quantity of our own work.

If we could put forward our work earlier than the deadline, then we'll be able to ask opinions on where to improve, quality or quantity as required.

Conclusion

All these tips might sound difficult, but for people in a junior position, it is much easier than trying to focus only on improving quality or quantity.

I think of my ability to learn things slower than other people and often find myself focusing too much on the quality. I was always in a hurry to catch up with other people.

Then I came across these principles in a book called "内定者への手紙(A letter for our new employees)" written by Yuiga Kitano and realized I was underestimating these essence.

Since I keep these three principles in mind and focusing on speeding up on tasks, I feel much more confident and have more time brushing up on my work.

As an apprentice, I am going to keep this mindset while I'm in on-the-job training so that I can grow as fast as possible.

Thanks for reading. If you have any opinion or question, please leave a comment below! In the meantime, please follow me on Linkedin @Maiko Miyazaki or Twitter @Maiko Miyazaki

Creating Class diagram - 3 basic steps to make it easier

Maiko Miyazaki — Sun, 28 Feb 2021 20:13:23 +0000

A class diagram could play a key role in a solid software design. However, I hardly knew where to start when it comes to the class diagram even though I take the time to analyze requirements.

In this article, we will discuss steps to discover the path to create a class diagram from a scenario.

To summerise, we will go through these steps:

0️⃣ Our scenario
1️⃣ Write user stories
2️⃣ Create CRC cards
3️⃣ Translate CRC cards into a class diagram
4️⃣ Conclusion

Let's get started 😊

O. Our scenario

Assuming we have a client who owns 5 cafes and a kitchen where kitchen staff make food items every morning for all shops. The system they need is food ordering software where each shop manager submits food orders for the next few days and the kitchen staff view them. We have a report that summarises objectives, criteria, all the requirements and limitations.

Our goal, for now, is to create a class diagram. But can we start drawing by looking at the report? As long as the report is well organized already aiming to create a class diagram, or if we are experienced, I assume we can. Otherwise, we may want to start by writing user stories that help us ensure to meet each requirement.

1. Write user stories

A user story looks like this:

As an admitted user, I want to add an item with quantity to my order list so that an admitted user can view them.

Point is to write from the perspective of end-users so that verbs are in the active forms. If we fix the sentence in the "As a ... , I want to ... so that ... ." form, we can find important information easily. "so that" part can be eliminated if the objective is obvious.

I personally like writing user stories on Evernote or Spreadsheet, because user stories can be moved around to organize with no hassle.

We can identify objects when looking at nouns in the user stories like this:

"As an admitted user, I want to add an item with quantity to my order list so that an admitted user can view them."

And verbs can help identify requirements that the object might have.

"As an admitted user, I want to add an item with quantity to my order list so that an admitted user can view them."

With this sentence, we can say that the order list object could have the responsibility related to add and view.

Assuming the above list includes all the requirements we need to implement. For now, we can simply highlight nouns and verbs, so we can look into details in the next step.

2. Create CRC cards

In this section, we'll organize the components that we just highlighted in the user stories. Here we use CRC cards.

CRC cards are a great way to refine and record the components of the system design. They are like a pre-class diagram. What we need are a pen and physical cards - I personally like to use large size post-it, but we could also use online platforms like Lucidchart.

A CRC card has three sections:

Class name
Responsibilities
Collaborators

Firstly, we'd like to find the class names. Let's look at the first user story on the list.

"As an admitted user, I want to add an item to item list. Each item is created with its name, description, production cost and availability."

In the above sentence, we can see three potential objects: (admitted)user, item, and item list which are all nouns. They can be classes, so create CRC cards with their class names.

The next section is Responsibilities. Here we can focus on verbs in the user stories and consider who is responsible for the action. For instance in the sentence, "add" is the responsibility of the class ItemList.

But when you think about it, it's a bit strange, right? Because in real-life, if you are a user of a box and if you want to put something into it, the activity itself is not the responsibility of the box but rather the users. So why the add is the responsibility of ItemList, not the User?

Because in object-oriented thinking, objects are all self-aware, even if it's not a living thing. So we think all objects can do something too.

Imagine our ItemList is a living thing that works very hard for you. If a user tells it to put something into it, they'll do it. If they can do it, you should leave the responsibility to them.

So the CRC card for the class ItemList can have a list of items and the ability to add an item into the list.

Lastly, we'll find Collaborators. For ItemList class, a collaborator can be Item because it's the object that will be added in the ItemList using the add activity. Therefore they are collaborators to each other.

While creating CRC cards, it is good practice to look at the collaborators' section and place the cards next to each other if they have a relationship. We can move around as you add new information.

3.Translate CRC cards into a class diagram

When we have covered most of the nouns and verbs on CRC cards, we can move on to translating them into a class diagram.

It's now much easier to draw it using both user stories and CRC cards. I personally like using Lucidchart that helps create clear diagrams easier.

For instance, I have translated the CRC card to the class like this:

Responsibilities in the CRC card such as its name, description, production cost, availability was included as attributes, and methods were evolved from edit each component responsibility. Ideas of relationships came from the collaborator section.

Make sure to cross out each user story as you create related class diagram components -- so that we won't miss out on any requirements.

If you have difficulties to identify Objects or relationships, I'll highly recommend taking a look at Object-Oriented Design course on Coursera -- You could even learn most of the module by just reading its class note pdf.

Conclusion

Creating a class diagram is something that can help us create a solid software design as well as to communicate with other members of the team or future self, but at the same time, we may not have much time to spend on this activity. Using these small techniques would help us achieve a better result at the end of the day.

Thank you so much for reading, if you have any thoughts, please leave a comment! In the meantime, please follow me on Linkedin @Maiko Miyazaki

How to create hourly weather auto-collector using AWS Lambda with Python 🐍

Maiko Miyazaki — Sat, 20 Feb 2021 12:15:44 +0000

In this post, we'll be creating an hourly weather auto-collector using AWS Lambda.

This Lambda function will collect the previous day hourly weather data from OpenWeatherMap API, clean it, then upload it on the S3 bucket.

The weather data will look like this:

And it will be done once a day, so we'll have yesterday's hourly weather data uploaded daily.

Assuming we already have:

AWS account
Have an S3 bucket where we will upload the data on
Free version of OpenWeatherMap API key

Steps we will go through

1️⃣ Create a policy
2️⃣ Create a role
3️⃣ Set up a Lambda Function
4️⃣ Write code to fetch, clean the data from OpenWeatherMap
5️⃣ Install libraries in the same directory
6️⃣Upload the zip file on the Lambda console
7️⃣ Set up CloudWatch
➡️ Done!

Let's get started 😊

1. Create a policy

To begin with, we first need to create a policy allowing a role to upload files on our S3 bucket.

Open your IAM console, and click Policies, then Create permission button.

On the next page, we'll be able to write a policy in JSON format. It should look like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "s3:PutObject",
           "Recource": "arn:aws:s3:::yourbucketnamehere/*"
        }
    ]
}

When it's ready, click Next: Tags button, then click the Next button to review.

Name the policy and click Create policy.

2. Create a role

Now we will create a role that we will attach the policy we just created.

On the same IAM console, click Roles, then Create role.

On the next page, select AWS services as the entity and Lambda as a use case. Click the Permission button to go to the next page.

On the next page, you can define what permission you are going to give to this role. Search the policy name you have given to the policy we just created.

Check the policy and click the Next button. Click the Next button again to name your role.

Then click Create role.

The Lambda function we will be developing in the next step will be allowed to upload files if this role is attached.

3. Set up Lambda Function

Next, we will set up our Lambda function.
Open the Lambda console and click Create button.

Select Author from scratch, and enter the Function name you like. In this tutorial, we will use Python3.8.

On the Permissions section, select Use an existing role. Then choose your role we just created. When it's done, we can click the Create Function button.

Now we could attempt writing function code on the page, but most of our essential libraries cannot be imported through the Lambda function.

Attempt to write code

Unable to import modules.

Therefore we will first need to create the lambda_function.py file locally and install the necessary libraries on the same directory.

4. Write code to fetch, clean the data from OpenWeatherMap

Ok, now we will write code to clean the weather data fetched from OpenWeatherMap API.

Create a folder locally with lambda_function.py file in it.

Step1: Get the data

We start by getting the hourly weather data in a certain location.

In your lambda_function.py file, firstly import all the necessary libraries and define lambda_handler function.

import os
import sys
import requests
import json
import pandas as pd
from datetime import datetime, date, timedelta, timezone
import boto3
import csv

def lambda_handler(event, context):
    ## We will start writing code here

Inside the lambda_handler function, we'll fetch weather data from OpenWeatherMap. To do so, firstly assign necessary information into variables.

    api_key = 'your_own_api_key_here'
    url = 'https://api.openweathermap.org/data/2.5/onecall/timemachine'
    yesterday = datetime.now() - timedelta(days=1)
    timestamp = round(datetime.timestamp(yesterday))
    params = {
        'lat': '53.349805',
        'lon': '-6.26031',
        'units': 'metric',
        'dt': timestamp,
        'appid': api_key
    }

The geographical coordinates are set to Dublin, Ireland. Please find out your location by googling longitude and latitude [your location]. South and East are set to negative.

Now we will send a request to get hourly weather data for yesterday.

result = requests.get(url=url, params=params)
result_json = result.json()

In the result_json variable, we should have:

{'lat': 53.3498,
 'lon': -6.2603,
 'timezone': 'Europe/Dublin',
 'timezone_offset': 0,
 'current': {'dt': 1613668586,
  'sunrise': 1613633820,
  'sunset': 1613670073,
  'temp': 6.92,
  'feels_like': 1.1,
  'pressure': 998,
  'humidity': 76,
  'dew_point': 2.99,
  'uvi': 0.84,
  'clouds': 75,
  'visibility': 10000,
  'wind_speed': 6.17,
  'wind_deg': 240,
  'wind_gust': 12.35,
  'weather': [{'id': 803,
    'main': 'Clouds',
    'description': 'broken clouds',
    'icon': '04d'}]},
 'hourly': [{'dt': 1613606400,
   'temp': 9.62,
   'feels_like': 4.5,
   'pressure': 991,
   'humidity': 81,
   'dew_point': 6.52,
   'clouds': 75,
   'visibility': 10000,
   'wind_speed': 6.17,
   'wind_deg': 160,
   'weather': [{'id': 500,
     'main': 'Rain',
     'description': 'light rain',
     'icon': '10n'}],
   'rain': {'1h': 0.51}},
  {'dt': 1613610000,
   'temp': 9.75,
   'feels_like': 2.49,
   'pressure': 987,
   'humidity': 81,
   'dew_point': 6.65,
   'clouds': 75,
   'visibility': 10000,
   'wind_speed': 9.26,
   'wind_deg': 170,
   'wind_gust': 16.46,
   'weather': [{'id': 500,
     'main': 'Rain',
     'description': 'light rain',
     'icon': '10n'}],
   'rain': {'1h': 0.89}}, .....

Step2: Clean the data

What we want to know is only the hourly data from the JSON data, so we get that part in weather_data variable using pandas.

weather_data = pd.json_normalize(data=result_json['hourly'])

Result:

However, what we want to get from this dataframe is only dt and feels_like, and the data that are still nested inside the 'weather' column, so we will change the above code into this:

weather_data = pd.json_normalize(data=result_json['hourly'], record_path='weather',
                                 meta=['dt','feels_like'])

Then we will get only the necessary data.

We can use these data as it is, but I chose to remove some of the columns as they are redundant.

weather_data = weather_data.drop(['main', 'description', 'icon', 'temp', 'clouds'], 1)

Now we get the minimum required data in the data frame.

We would also like the dt column as easy-to-read format, so we will change it too.

weather_data['dt'] = weather_data['dt'].apply(lambda x: datetime.fromtimestamp(x))
# we will also assign date as a part of file name later on.
date = weather_data['dt'][0].strftime("%m-%d-%Y")

Result:

We could also change the format to "%m/%d/%Y %H:%M:%S" if you'd like.

weather_data['dt'] = weather_data['dt'].apply(lambda x: x.strftime("%m/%d/%Y %H:%M:%S"))

Result:

In this tutorial, we don't want the data from 0:00 to 5:00, 21:00 to 23:00, so we'll get rid of them as well.

weather_data = weather_data.drop(weather_data.index[21:])
weather_data = weather_data.drop(weather_data.index[:6])

Result:

Step3: Write a file on the S3 bucket

Now we will write the data into a file and upload it on the S3 bucket.

// Convert the data frame into CSV
csv_data = weather_data.to_csv(index=False)

s3 = boto3.resource('s3')
bucket = s3.Bucket('your_bucket_name_here')
key = '{}.csv'.format(date)

with open("/tmp/{}.csv".format(date), 'w') as f:
    csv_writer = csv.writer(f, delimiter=",")
    csv_reader = csv.reader(csv_data.splitlines())
    for row in csv_reader:
        # each row looks like this..
        # ['id', 'dt', 'feels_like']
        # ['801', '02/18/2021 06:00:00', '-2.49']
        # ['801', '02/18/2021 07:00:00', '-1.84']....
        # write each row on f using csv_writer
        csv_writer.writerow(row)
bucket.upload_file("/tmp/{}.csv".format(date), key)

5. Install libraries in the same directory

In AWS Lambda, many libraries cannot be imported therefore we need to have them in the same directory where we have the lambda_function.py.

For our lambda function, we need to have NumPy, Pandas and Requests installed.

I have found this article extremely helpful, so please have a look if you'd like to know the way step by step.

After installing all the libraries, we need to compress all the file. I have mine as archive.zip but the name doesn't really matter.

6. Upload the zip file on the Lambda console

In your lambda console, we'll be able to find Upload a zip file button inside the Actions dropdown. Up load your zip file from there.

When it's done, we can run a test from the Test button that is located top-right of the page. You'll need to configure the test event, but you don't have to do much here, so just name the test and hit Create. Then hit the Test again.

Sweet! It says the test has run successfully.
Let's see if the CSV file is correctly saved in the S3 bucket.

It seems the file is uploaded correctly.

7. Set up CloudWatch

Setting Cloudwatch for our lambda function enables the function to run automatically.

Let's open the CloudWatch console. Click the Create rule button.

In the Event Source section, select Schedule and set our desired interval. I'll set it to run once a day.

In the Target section, select Lambda function and choose our function name from the list. Hit the Configure details button.

Name your CloudWatch rule on the next page, and hit Create rule button.

Perfect!

Check if your function was run as soon as you created the CloudWatch rule as well as running as your expected interval.

Complete code in the lambda_function.py

import os
import sys
import requests
import json
import pandas as pd
from datetime import datetime, date, timedelta, timezone
import boto3
import csv


def lambda_handler(event, context):
    api_key = 'your_openweathermap_api_key_here'
    url = 'https://api.openweathermap.org/data/2.5/onecall/timemachine'
    yesterday = datetime.now() - timedelta(days=1)
    timestamp = round(datetime.timestamp(yesterday))
    params = {
        'lat': '53.349805',
        'lon': '-6.26031',
        'units': 'metric',
        'dt': timestamp,
        'appid': api_key
    }
    # Fetch hourly weather data in Dublin from OpenWeatherMap API
    input_file = requests.get(url=url, params=params)
    result_json = input_file.json()
    # Flatten and clean hourly weather data
    weather_data = pd.json_normalize(data=result_json['hourly'], record_path='weather',
                                    meta=['dt', 'temp', 'feels_like', 'clouds'])
    weather_data = weather_data.drop(['main', 'description', 'icon', 'temp', 'clouds'], 1)
    weather_data['dt'] = weather_data['dt'].apply(lambda x: datetime.fromtimestamp(x))
    date = weather_data['dt'][0].strftime("%m-%d-%Y")
    weather_data['dt'] = weather_data['dt'].apply(lambda x: x.strftime("%m/%d/%Y %H:%M:%S"))
    weather_data = weather_data.drop(weather_data.index[21:])
    weather_data = weather_data.drop(weather_data.index[:6])
    csv_data = weather_data.to_csv(index=False)

    #call your s3 bucket
    s3 = boto3.resource('s3')
    bucket = s3.Bucket('your_bucket_name_here')
    key = '{}.csv'.format(date)

    with open("/tmp/{}.csv".format(date), 'w') as f:
        csv_writer = csv.writer(f, delimiter=",")
        csv_reader = csv.reader(csv_data.splitlines())
        # Iterate over each row in the csv using reader object
        for row in csv_reader:
            # row variable is a list that represents a row in csv
            csv_writer.writerow(row)
    #upload the data into s3
    bucket.upload_file("/tmp/{}.csv".format(date), key)

Thanks for reading!

If you have any ideas to improve the function, please leave your view in the comment! I would truly appreciate it 😊 In the meantime, follow me on Linkedin @Maiko Miyazaki

Resources

AWS Lambda with Pandas and NumPy by Ruslan Korniichuk
AWS Lambda with Pandas and NumPy|Pandas & AWS Lambda|Pandas Lambda with Python3 by BidDataOnlineSchool

First meeting with a client? Knowing requirement gathering techniques makes a difference

Maiko Miyazaki — Sun, 14 Feb 2021 18:53:02 +0000

When we have a project to develop a system, we would be gathering requirements before diving into feasibility study. Whether it is for a client or for ourselves.

Assuming we have received an email from an education provider saying that they are looking to develop their own application system and asking if we can develop them. The first thing we want to know is if that's feasible to develop, so we sent them back an email to schedule a meeting in several days.

At this point, let's say we already have some ideas on the outline of the system and get excited about the project. Probably, we'll be saying in our mind, "Oh, maybe we can use the new technology that is popular these days." or "Perfect! This is a great opportunity to use the programming languages that we've been interested in".

So, what will we do in the first meeting? Will we tell them about our ideas and what we are capable of?

Not quite. That's actually what we should avoid doing. What we must to do is to focus on being an active listener and gather all the information including underlying needs. Requirements gathering techniques will help us achieve it.

But why do we have to do that? Why not jump into the feasibility study right away?

Reasons for running requirement gathering

There are many reasons that we need to allocate time for gathering requirements.

1. Help to discover important requirements that could influence the development cycle.

Imagine we decided to start designing the online application system for the education provider mentioned earlier. The head of the organization told us to implement specific functionalities, so we simply implemented them.

But what if she/he didn't understand what the senior management team really needs? What if we discover that the system is being used mostly by the management team, then we need to align the project with the management team's need after deployment?

It's not always the case that essential requirements are always evident and visible, that's why we need to take time to gather requirements from all users.

2. Solve possible misinformation or misconception

We, humans, are not perfect. Especially when it comes to understanding others, it requires certain skills and techniques.

As English is not my first language, I often find myself misunderstanding what others really meant more often than when I communicate with others in Japanese. So, at least to me, when misunderstanding each other, we have a different definition of certain words in mind.

This is applicable especially in the IT field where you communicate with people who do not have the same understanding of technology.

Assuming we are aiming to create a sensible system for their business, then it's not for ourselves, therefore we need to avoid misunderstanding what the client means at all cost by having jargon-free conversations.

Because small things add up to a big difference.

3. Put their mind to the systems development.

It is important the client feel they are heard and understand that their involvement and voices are crucial for the successful project.

For instance, if we talk too much about our vision or explaining about new technologies instead of listening to the client, they are more likely to feel that their opinions are not in the highest priority, or they may conclude that they don't need to put their mind to it because we sound like knowing enough about the project.

Requirement gathering techniques can help us to avoid deepening the gap between the client and the development team's perspective and motivation.

To sum up, the effect of gathering requirements should not be underestimated and the techniques should be learned and used effectively.

Technique 0: Preparation

Now we will discuss the techniques useful for the meetings and interviews. The first thing is preparation.

What we do here is to research the client as much as possible. When you think about an interviewer you believe they are highly skilled, you may surprise how much they have given research about the interviewee. If I were to be interviewed by them, I would be more open-minded to answer questions.

To achieve the same effect on our client, we could gather information online but recommended to review documents if they are available from the client.

What we aim here is to examine:

Company's objectives and goals
Organization structure
Their information systems

Getting ideas about these topics will enable us to identify what kind of questions we should ask the specific person.

After the research, interview questions can be listed and organized carefully, so the questions are complete and can be answered without confusion.

Technique 1: Open-Ended Questions

Open-Ended questions let the interviewee talk about the topic freely rather than providing a specific answer. For instance, questions such as "What are your reactions to the current/new online application system?" can be one of them.

This style of questions is especially helpful if we want to delve into the topic and look into the underlying issues.

Technique 2: Closed Questions

Closed questions, on the other hand, is the way to clarify the ideas and specific answers. Questions can be answered with "Yes/No", "Agree/Disagree" or specific numbers.

For example, "Do you agree or disagree that the current system is obstructing your productivity?" can lead the interviewee to one of the other answers.

We can also alter the above question to be more specific, by asking "What specifically is the problem on the current system that is obstructing your productivity?".

This can be altered depending on how much you already asked or understand the interviewee's perspective about the topic.

Disadvantages of Open-Ended and Closed questions

There are possible disadvantages if we use only one of the other style of the question mentioned above.

If Open-Ended questions make up the majority of the questions, it is harder for us to control the interview. Moreover, we may not be able to obtain a sufficient amount of information compared to the time we spent on the interview.

Oppositely, only asking closed questions may fail to take out detailed information because they could answer single word if we don't delve into it.

As far as I imagine, it's rare to be using only closed questions in a real conversation, so most of us use mixed styles or open-ended style only.

Technique 3: Pyramid Structure

In order to prevent falling into a vicious cycle of interviews or meetings, we need to organize the order of questions depending on who we are interviewing.

Pyramid Structure is one way to organize them. Under this method, the sequence of questions starts from very detailed, often closed type. And then we can gradually broaden the topics.

If the interviewee is the lower-level users in the current system, then they might need this structure of questions to warm up to the topic.

A good example is questionnaires, where the level of understanding about the topic may not be the same among respondents.

Technique 4: Funnel Structure

Not like the Pyramid, Funnel Structure can start with general questions and gradually narrow the topics, and conclude with the closed type of questions.

This method is especially effective for an interviewee who could have deliberated the topics already. It's likely that the person you see at the first meeting has clear opinions about the topic already, so it might be a good idea to structure questions funnel style.

Technique 5: Modelling

Because the requirements can be complicated, it is often difficult to see each component and relationships in a big picture. Especially for the client, it can be challenging.

Modelling is a diagrammatic way of representing system components enabling us to study the requirements intuitively.

UML (Unified Modeling Language) is a common way to create many styles of diagrams in a clear manner. The above image is called Use Case Diagram, which can be created to clarify how different types of users interact with the system.

In other words, we can summerise the information we collected during the interview by creating diagrams. When it's done, It can be investigated by clients and the development team.

Conclusion

Requirement gathering techniques are mostly about active listening skills, and how we could align perspectives of the client and ourselves. This can be challenging.

But if we could cover all of the mentioned above, we would be more confident about what we are doing, getting trust from the client while creating a solid base to build a successful project on top of it.

Resources

Open-Ended and Closed Type Interview Questions
All You Need to Know About UML Diagrams: Types and 5+ Examples
Object-Oriented Foundations - Linkedin Learning

3 Main Benefits of Feasibility Study

Maiko Miyazaki — Sun, 07 Feb 2021 16:51:30 +0000

Introduction

I am participating Female Apprenticeship programme coordinated by FIT since December 2020, and I've been writing posts about what I've learned in the Off-the-job training. In this post, I am going to summarize the main benefits of feasibility study which is a part of the first phase of the software development life cycle.

Benefit 1: Avoid risks to fall a project by the wayside.
Benefit 2: Avoid reinventing the wheel.
Benefit 3: Clarify tasks to discover requirements.
Conclusion

Benefit 1: Avoid risks to fall a project by the wayside.

Whether or not we are developing our own personal project or an app for a client, there are a number of risks that'll lead us to give up the project itself. Running feasibility study should play a key role here.

Economical issues

Economical issues are one of the factors that may force us to cease the project. Let's say, for example, we've been asked to develop a website for your friend, Bob's restaurant, for free. We readily answered that we can do it since we were just beginning to learn web development, so we thought it would be a great opportunity to upskill. We began to create it on our computer locally right away.

However, you began to realize that running website costs some money for web servers, domain name and features that Bob requires. So you decided to ask him if he would be able to pay for it, but he replied to us that he cannot afford any of it. What you can do now is to pay it with your money or cease the project.

If we could run a feasibility study at the very beginning, we could take measures to make a better choice.

So when running the study, we need to ask:

What are the required costs?
How much is the budget?
If this project fails, could it put the business at risk?
What is the expected benefit regarding revenue?
Is it a good investment overall?

Technical Issues

Other potential risks can arise from technical issues. After few days Bob told us that he can't afford running the website,
he came back and told us that he spared his money. So we restarted creating his website from where we left off.

We kept working on the project and finally, it was successfully released online. What an achievement!

However, we didn't know the server can be down if the website got many visitors more than it can manage. The tragedy happened because Bob's restaurant was so popular in recent years so the website became viral very very quickly. Now we need to change the webserver that can maintain the number of visitors that the website has, but it costs some money and time.

If we could run a feasibility study at the very beginning, we could choose a suitable server in the first place.

Therefore, when running the study we need to list up obligations which can be written with should/must. For instance:

The website must be able to maintain .. visitors at once.
Users should be able to send a message from query form.
_The website must transfer the messages to a certain email address. _

Operational issues

Like the factors mentioned above, the operational problem should be discovered before we began planning the project. For instance, if we are to develop a sales forecast app for a client who has 5 cafes in town, and they want a system to help each shop to bake pastries just right amount in the right time.

Assuming we successfully developed and deployed the project but after several weeks we realized that they rarely use the app. The reason was that each shop was very very busy and they only have just enough staff, so no one could manage to go to the office to open the app and check the forecast when they are running for the customers.

If we could run a feasibility study at the very beginning, we could realize this would happen.

Like the technical issues, we need to list up obligations which can be written with should/must. For instance:

The system should be used on a tablet or smartphone easily.
The system must be intuitive to understand how to use.

Benefit 2: Avoid reinventing the wheel.

In the feasibility study, we can discover resources that can be used for the project. For instance, we need data about sold out items when developing a sales forecast application, so we implemented a form for users to let the system know that the item was sold out. But later on, we discovered that they've been collecting information about soldout items on their cashier system. We could actually fetch the data from there rather than implementing the feature from scratch.

Inventing the wheel might bring advantages, but it is depending on whether it is intentional or not.

On top of that, investigating available resources is vital because we have a deadline in many cases, therefore creating what is already out there should be often avoided. As a result, we will be able to spare time to focus on high-risk requirements.

In order to identify what's available, we could ask:

What the flowchart of the current system looks like?
Where does the current system collect certain data from?

Benefit 3: Clarify tasks to discover requirements.

Because we now have an exhaustive list of issues is documented, required information and solution are easier to be identified in the next step. In other words, without the feasibility study, we may be less likely / less confident that the requirements are all-inclusive.

Conclusion

Feasibility Study is one of the things I wish I knew about at the very beginning of my freelance web developer journey. I've created many websites for clients and personal projects while teaching myself how to code, but I had always a tough time when it comes to planning and all the process onwards. Shame on me -- I didn't know how vital to take time in understanding if the project itself is worth going forward in terms of technical, operational and economical perspectives. That's one of the reason many of my projects failed to reach deployment. So I believe that knowing why the feasibility study is crucial can be a great starting point to successfully develop projects.

This investigation can be done as a preliminary investigation, so you could do it at the very first meeting with your client but I would also do it when I come up with personal project ideas.

Resources

System Analysis & Design - System Planning - tutorialspoint.com

Thanks for reading until the end!
If you have any questions or opinions, please leave a comment 😊

Why SDLC? -- Things I've learned in off-the-job training

Maiko Miyazaki — Sun, 31 Jan 2021 09:33:11 +0000

Introduction

I am participating Apprenticeship programme coordinated by FIT since December 2020. In the programme, we learn about software development in a college for the first six months (we call it off-the-job training. )and moving onto on-the-job training several days a week.

If you are interested in how I got hired as an apprentice by a financial service firm in Ireland, please refer to my post.

At the moment, we are on the module of Systems Analysis and Design and I am going to summarize what I've learned so far.

Why SDLC and who should learn them?

The Software Development Life Cycle is a general term for many SDLC methodologies which are approaches to tactically develop high-quality software.

Developing a piece of software without strategy is like trying to solve an issue while you don't know what it is nor how to solve it. We may think that we know what we want to create and might be possible to jump into coding without a strategy, but along the way you are likely to face issues such as unexpected cost, prolonging schedule or even loss of motivation.
Understanding SDLC is never more important if we are working in a team, but also on your personal project.

So if you are a self-taught developer who

..wants to develop a personal project successfully

..is working as a freelance and want to gain trust from clients

..is stuck with your own project because of the budget issue or loss of motivation

..started to look for a software development job

..wants to develop software in a team

Learning SDLC methodologies will help you significantly.

What is SDLC like?

All SDLC methodologies not only but focus on these phases:

1) Plan, gather and analyse
This phase answers questions like...

- What are the objectives?

- Who uses the system?

- What functionality do we need?

- How the users use the system?

- Are they feasible to implement? ...etc

2) Design

- How will the software work?

- How will it look? ...etc

3) Implement

- How can they be programmed?

4) Test

- Does each unit work?

- Does each unit work when combined as a group?

- Does the full system work when all units are integrated?

- Can the system support the business and users in a real situation? ...etc

5) Deploy

5)Maintenance

Each methodology approaches these phases differently, but the basic idea about each of them are generally similar.

What are the basic SDLC Models?

There are many more models out there, but in this section, I am going to summerise 4 types of SDLC models namely Waterfall, V-model, Spiral and Agile.

Waterfall Model(1956~)

Waterfall methodology is one of the oldest and simplest one amongst the others. It is categorised as a linear-sequential lifecycle model, which means the whole process is divided into separate phases, and each phase of the development has to be completed before moving onto the next one.

Like the name “Waterfall”, the flow of the development is moving from the top to the bottom -- from analysing requirements to deployment and maintenance phase -- and it doesn’t allow us to go back to the previous phases even if we want to add or alter a requirement. So if we need alteration, we need to start from the very beginning. That being said, we will not be able to see the prototype or working software until the very end of the project, but the advantage is the ease to manage and monitor the whole process for the development team.

If all the requirements are very clear and unlikely to change over time, this model enables us to deploy the system in an efficient manner, however, it often not applicable for the larger projects where not all of the requirements might not be transparent in the beginning.

Used in manufacturing and construction industries

Reference:
Waterfall model - wikipedia

V-Model(1980s)

The V-Model is similar to the Waterfall Model since it is a sequential type in which each stage begins only after the previous stage has ended. It also has the same development cycle as the Waterfall, however, each cycle includes planning for tests associated with each phase. In other words, the corresponding testing phase of the development phase is planned in parallel.

Like the Waterfall Model, the steps go through the business requirement analysis, System design, Architecture design, Module design, and then the team dives into the coding phase. After completing the implementation, it goes through all the testing phases that had been planned beforehand.

This methodology allows the management team to decide everything before getting into the coding phase, more precisely than the Waterfall model. In order to function this model, the clarity of the requirements is also vital like the Waterfall Model. For this reason, the V-Model can be only ideal for the smaller projects but can be problematic to apply if the project requires flexibility for the requirements.

Used in medical development field

Reference:
SDLC - V model - tutorialspoint
V-model - wikipedia

Spiral Model(1986)

Unlike these two models above, the Spiral Model does not follow sequential steps but iterative development, which begins by specifying and developing just a part of the system and iteratively enhances the evolving versions until the full system is implemented, thus it won’t be an issue if not all the requirements are crystal clear at the beginning of the process. For this reason, this model can be plausible when the customer or the owner of the system has only a vague idea of the requirements, or the requirements are complicated.

The advantage of the Waterfall Model - systematic control aspect - is included in this model which is the Identification of a requirement, Engineering, Evaluation and Risk Analysis. Compared to the sequential type of models, the Spiral Model requires customer evaluation in the last part of each iteration, so that customer needs to be able to involve in the development more frequently than the former two models.

A typical iteration is somewhere between 6 months and 2 years and it can also be a long-term commitment owing to the addition or changes for requirements overtime. This flexibility can be a disadvantage as the team might be unable to estimate the end of the project, and it requires more attention from the management team as this structure is more complex than the former two models.

Used to develop the early version of Windows in Microsoft.

Reference:
Difference between agile model and spiral model - stackoverflow
What is Spiral model- advantages, disadvantages and when to use it? - TRY QA
Spiral Model SDLC: Top Choice for Large, High-Risk Software Projects? - intellectsoft

Agile Model(2001)

The last model that I am going to mention is Agile, which is a combination of the iterative process like the Spiral model with a focus on process adaptability. It produces a working software product at the end of each iteration which may not be true for the Spiral model.

Every iteration of the Agile model involves cross-functional teams working simultaneously on various areas and these teams are identified as Planning, Requirement Analysis, Design, Engineering, Unit testing and Acceptance Testing. Because of the structure of the model, it doesn’t always require a concrete plan or requirements at the beginning but develops a system through repeated cycles in smaller portions at a time. This style enables us to prioritize developing working systems rather than creating concrete documentation and to be flexible for change rather than following a concrete plan.

Every iteration of the Agile model is typically shorter -- 2 to 4 weeks in most cases, therefore you’ll be able to see and monitor the progress through the working system more frequently. That being said, it can be harder to maintain the development because of the adaptability, therefore a management team plays the key role to maintain the direction of the team at all times. Also, each team must be self-organized, highly collaborative and accountable for their own work.

Used on nearly any large scale project in any industry

Reference:
Software Engineering | Comparison between Agile model and other models - geeksforgeeks
Agile software development - wikipedia
Who Uses Agile? - Wrike

Conclusion

To summarize, the Waterfall and V-Model require strict planning at the beginning of the project, therefore it is often not suitable for many systems that are going to be used in a dynamic environment. This is one of the reasons for not being popular in recent years. On the other hand, the Spiral model or Agile Model can be more sensible for many situations unless you have developed exactly the same system in the past. So, if I were to build my own development team or develop a software for a client I would adopt Spiral or Agile methodology. Comparing these two methodologies, I would prefer Agile since it lets the team focus on customer satisfaction which I always believe is most important.

Do you agree? Please share your thoughts in the comments!

Forem: Maiko Miyazaki

How to create a related post component with gatsby-remark-related-posts plugin

Motivation

Requirements

Implementation

gatsby-config.js

gatsby-node.js

templates/article.js

Consideration

20 things I learned about Scrum from Learning Agile

1. Each role has a different perspective on user stories

2. Before diving into coding, have discussions to see user stories in the shoes of other roles

3. It's not enough to have the "correct" process or the "best" practice

4. Focus on working software rather than comprehensive documentation

5. The term "Working software" has a specific definition

6. Be cautious when you are constantly "negotiating" to protect yourself

7. Constantly look for changes

8. Principles add life, character and heart into Practices

9. Scrum Master doesn't own the plan

10. Product Owner makes decisions on behalf of the business

11. Programming isn't the only important thing in a project

12. Be cautious when prioritising your own professional goal

13. The more people outside the team involved, the more value he or she can potentially add

14. Collective commitment rather than committing to individual tasks

15. Task assignments come from the team member themselves

16. Daily Scrum is not a way for the team lead to manage the schedule

17. Ask each other 3 questions every day on the Daily Scrum

18. Pleasant surprises for users/stakeholders can be almost as damaging as unpleasant ones

19. Have a concrete definition of "Done"

20. A lifetime to master

C# date & time conversion CheatSheet

Table of Contents

String to DateTime object

String that is not parsable with Parse() to DateTime object

DateTime object to String

String to String

DateTime Object to Unix Timestamp in Seconds

Unix Timestamp in Seconds to DateTime Object

UTC timezone String to DateTimeOffset object in a different timezone

24-hour time String to 12-hour time String

Download PDF file

Resources

Python date & time conversion CheatSheet

Table of Contents

String to DateTime object

DateTime object to String

String to String

UTC timestamp in seconds to DateTime object

UTC timestamp in seconds to String

UTC time string to UTC DateTime object in different GMT

24-hour time String to 12-hour time String

Download PDF file

Let's remember the concept of Abstraction, Generalisation, Decomposition and Encapsulation in one go!

Summary

Abstraction

Generalization

Decomposition

Composition (with Dimension)

Aggregation (with Tap, Ventilation fan, Oven...)

Association (with Designer)

Encapsulation

Conclusion

References

Newbie's Cheatsheet: Commonly used verbs for naming functions, methods and variables

Table of Contents

1. Alteration

2. Creation

3. Establishment

4. Obtainment

5. True or False Statement

6. Conclusion

Focus on being FAST at getting work done rather than Quality or Quantity.

Why SPEED is the key?

How to SPEED UP?

1. Provide an answer right away

2. Start right away

3. Put forward earlier

Conclusion

Creating Class diagram - 3 basic steps to make it easier

O. Our scenario