dailysoftwareengineerinjapan - Tumblr blog

[dailysoftwareengineerinjapan]

Test Phase

We have entered the fourth test phase.

We made some parts of tests be able to run automatically. It is good. Especially, it is more efficient to automate the simple tests that it takes a long time. On the other hand, we run the complicated tests manually.

Although there are cumbersome works as like preparing test scenario and test data, it is not dislike to run tests manually for me.

I feel good when I confirm that our application runs as expected and I am proud of it.

Checking UI looks of an application, its intelligibility, message validity, and so on can't automate.

However, AI may run such works instead of us. I would happy if it could also register the issue to JIRA.

[dailysoftwareengineerinjapan]

Foods and Drinks Variation

Since I started working from home last year, I always drink coffee at home. Recently, we consume coffee beans with 400g by a month. I went to a coffee shop at last weekend and just bought 200g each of Paradise Premium and Kalosi Toraja.

However, I only drink a cup of coffee on weekdays. There is no special reason for it, but I can satisfy a cup of coffee in a day. I'm brewing coffee when I start work. After that, I have a ginger tea or a tea of chamomile and mint.

Previously, I rarely didn't have a hot tea at home. However, the drawback of working at home is that it is easy to get bored. Therefore because of avoiding it, I want various drinks. Next, I think about a snack variation. I want to choice better for my health like walnuts and cashew nuts.

Started working from home gives me an opportunity to re-think about foods and drinks. I think that it is good for me.

[dailysoftwareengineerinjapan]

Re: severe abdominal pain

Today also, I felt severe abdominal pain before lunch. This is a same pain that I felt last week.

I thought that the cause of the severe abdominal pain in the last week is to eat a pizza delivered at a previous day in the morning. However, I didn't eat them a few days.

The thing that it is same the last week is that today is Monday and yesterday is Sunday. Is it Monday sickness? However, my work is not so difficult as to feel depressing. Especially, the last Monday was calm because there are no MTGs.

We usually doesn't have a dinner because we have a lunch later than work day and we have a lunch fully. However, I may that this is not good. After we are hungry a long time, I may feel that when we eat something in the morning. As a trial, we will have a light supper next weekend. Another cause that I thought, I couldn't sleep well yesterday. The reason is that I slept until noon yesterday. So it may have caused a physical disorder.

If so, I should wake up at the time when I always get up in my workday. By the way, it is easy to remember the past things since continuing to write a diary every day.

[dailysoftwareengineerinjapan]

New DVD Recorder

Since new DVD Recorder was arrived at home yesterday, I have set up it today. Since new one is made by the same company as old one, it was so easy to wire the cables again. However, there was a problem also. We found that the BS / CS cable connects the TV directly without going through the old DVD Recorder. Therefore I have fix this by making go through the new one.

To move the contents that my wife recorded so far from old one to new one, I connected the both recorders to LAN with the switching hub and LAN cables. At our home, LAN cables are wired from the router to each rooms and there is a LAN cable connector close to an electronic outlet. When I connected it to the switching hub with a LAN cable, I found that the signal didn't come to the hub. So I checked the back panel of the router. As a result, I found that the router didn't connect to it. After I have fix it, the signal came to the hub.

I connected the old DVD Recorder to TV again to set up for networking. As a result, I could move one content of them to new one. When I tried to start all contents to move, it estimated that it will take 134 hours. So my wife seems that she is selecting a content with the highest priority and moving them one by one.

[dailysoftwareengineerinjapan]

Do my best

Today was the last day of current sprint. And it is also the review day. We have finished almost our tasks.

Although there are many differences comparing to implementing new feature, we did my best for them.

I will take a good rest on the weekend. And I decided to do my best for the next sprint.

[dailysoftwareengineerinjapan]

Paleolithic UI Programmer

UI fix Sprint will be about to end. I had some thoughts while fixing around HTML / CSS in this sprint.

display: flex is very useful. Even if we don't use a modern UI framework, we don't have to be depressed with saying 'Do I have this "div" elements side by side again?'

SVG is also good. We can adjust the size on displaying them. And we can change the background color with a text editor.

SCSS is nice for writing DRY CSS code. However, since it starts to convert to CSS on a small modification, it takes a long time to adjust styles.

Chrome's DevTools is great. When we want to change styles a little, we can change on this immediately.(Every modern browser has a tool like this...)

If we write 'cursor: pointer' for an element, a mouse pointer changes to a finger figure when it is on the element.(Am I a Paleolithic programmer?)

That's all. These are my impression who usually don't write HTML / CSS code.

[dailysoftwareengineerinjapan]

New Speaker for PC

I am listening music while working at home. So far I was listening with Google Nest mini that is a little far side from my desk. Because it is so cold recently, I use a hot air fan heater in my room. It is hard to listen music for me because the sound of fan is loudly.

So I bought a speaker for PC. It is better for listening music because it is in front of me and the sound reach straightly to my ears. And it has a dial that can control a volume. If I feel hard to listen music, I can up it immediately.

Recently, I am listening music on Sound Cloud. Since a volume of each contents is a little difference in it, it is helpful for controlling a volume immediately. It seems to make me comfortable on working time.

[dailysoftwareengineerinjapan]

Setsubun

Probably because I went to bed immediately after I took a bath, I didn't become my foot hot while I slept yesterday. However, it is hard to finish all things that I have to do before going to bed. I should remove something from those.

By the way, today is 'Setsubun' in Japan. It means before day of 'Risshun'. It means the day that it takes 1/8 year from winter solstice. We sprinkle roasted beans to a man dressed as a demon today. Then we eat those beans the number of ages. Those actions are thought as apotropaic in Japan.

Recently, 'Eho-maki' also becomes popular as much as 'Setsubun'. It is to eat a seaweed roll towards a certain direction. Today's our dinner is also those. The content of the seaweed roll were salad, tuna, salmon, etc. It was so good, but maybe I ate too much...

[dailysoftwareengineerinjapan]

Still I feel sick...

I went to a toilet a few times since I felt severe abdominal pain before lunch. After that, I had a pain in my shoulder blades and backache so I couldn't sit my chair.

I thought I started to catch a cold. So I measured the heat, but it was in the normal range. Immediately I took 'Kakkontou' (it is a kind of kampo) and I put a compress to my waist. After a while, I got better. I think that my severe abdominal pain is caused by eating some pieces of pizza that was delivered yesterday. The pain of my shoulder blades continue from last week.

I remembered that there are some different points from last week to this week. The one of them is to become my foot hot in sleeping. I looked it up on web , this seems to be caused by fatigue, cold body, disturbance of autonomic nerves, etc. I am going to take a bath today to improve blood circulation before going to bed.

[dailysoftwareengineerinjapan]

Out of order

Today is also day off. Because our DVD Recorder have been out of order a bit from last week, we went to Bic Camera to check a new one with my wife. We decided to buy a new model of the DVD Recorder we are using now because we have to move the contents that she recorded so far to new one.

The shop assistant let me know the way to wire around it for changing the new one. It will come to us next week. When it comes, we will start to wire them and do the setting hard. Just in case, I took a picture of the way to wire them displayed at the sales floor.

And he also let me know the way to copy all recorded contents from old to new. Both devices need to connect LAN of my home. They can't use wireless. So I bought the three LAN cables and a switching hub. This is the highlight of the software engineer's skill. I wish to copy all contents to new one with no trouble.

[dailysoftwareengineerinjapan]

CSV to Parquet

Today is day-off, so I prepared the data in order to run a benchmark test of my query engine.

I was trying to convert CSV to Parquet using Pandas.

import os import pandas from zipfile import ZipFile files = os.listdir("/var/data/csv") for file in files: if file[-4:] == '.zip': yyyy_mm = file[-10:-4] print(yyyy_mm) zip_file = ZipFile("/var/data/csv/" + file) dfs = {text_file.filename: pandas.read_csv(zip_file.open(text_file.filename)) for text_file in zip_file.infolist() if text_file.filename.endswith('.csv')} for k, df in dfs.items(): df.to_parquet("/var/data/parquet/" + yyyy_mm + ".parquet")

Each zip file contains several files, but only one CSV. So I thought that I could convert with this code (as above). However after it converted some files, it was crushed.

005_10 sys:1: DtypeWarning: Columns (37) have mixed types.Specify dtype option on import or set low_memory=False. Traceback (most recent call last): File "<stdin>", line 8, in <module> File "/usr/local/anaconda3/envs/to-arrow/lib/python3.7/site-packages/pandas/util/_decorators.py", line 199, in wrapper return func(*args, **kwargs) File "/usr/local/anaconda3/envs/to-arrow/lib/python3.7/site-packages/pandas/core/frame.py", line 2463, in to_parquet **kwargs, File "/usr/local/anaconda3/envs/to-arrow/lib/python3.7/site-packages/pandas/io/parquet.py", line 397, in to_parquet **kwargs, File "/usr/local/anaconda3/envs/to-arrow/lib/python3.7/site-packages/pandas/io/parquet.py", line 152, in write table = self.api.Table.from_pandas(df, **from_pandas_kwargs) File "pyarrow/table.pxi", line 1479, in pyarrow.lib.Table.from_pandas File "/usr/local/anaconda3/envs/to-arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py", line 591, in dataframe_to_arrays for c, f in zip(columns_to_convert, convert_fields)] File "/usr/local/anaconda3/envs/to-arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py", line 591, in <listcomp> for c, f in zip(columns_to_convert, convert_fields)] File "/usr/local/anaconda3/envs/to-arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py", line 577, in convert_column raise e File "/usr/local/anaconda3/envs/to-arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py", line 571, in convert_column result = pa.array(col, type=type_, from_pandas=True, safe=safe) File "pyarrow/array.pxi", line 301, in pyarrow.lib.array File "pyarrow/array.pxi", line 83, in pyarrow.lib._ndarray_to_array File "pyarrow/error.pxi", line 84, in pyarrow.lib.check_status pyarrow.lib.ArrowInvalid: ('Could not convert 1338 with type str: tried to convert to double', 'Conversion failed for column WheelsOff with type object')

It looks like the file contains an unexpected type of value... I will fix this tomorrow.

[dailysoftwareengineerinjapan]

At BURGER KING

I have also been to our office because the server that I maintained on last monday has a problem. When I was having a hamburger at Berger King on lunch time, old man and woman entered this shop. The woman sat down on a counter seat and the man started to order in front of cash register.

The counter was supposed to sit with one seat open to keep the distance between the guests for COVID-19. If it has seven seats, you can only sit in the first, third, fifth, and seventh seats. However, I think that the man will sit down the seat next to the woman because they are a couple and there are no guests. He sat down the seat after next the woman and placed their dishes on the table amoung them. Then they started to eat them without chatting. Even though they chatted friendly before ordering, they were completely silent during the meal.

This may be the correct behavior because COVID-19 is widespread. They followed a rule of this store completely. However I feel sad that they couldn't have a lunch with sitting side by side…

[dailysoftwareengineerinjapan]

Our CI / CD Pipeline

Our company has own CI/CD pipeline (We don't use CD for some reason…). When we create new git repo, it can collaborate the pipeline. We have also JIRA and Confluence. It is a great that we can fully use such infra.

If there were no CI/CD pipeline, I don't think that we want to write test code. Because when I feel like to run tests, I will find many errors. If we have no infra for DevOps when we start to develop new application, we should start to build it first. And it is good to have 'Backlog' to manage the requirements and 'Kanban' to manage our tasks.

I got a problem with CI today. Even though I have to solve this, I think we are blessed that there are such infra in our company. I appreciate the maintainers of those.

[dailysoftwareengineerinjapan]

How to fix my eyestrain

I have fixed some UI today. Although I am not good at HTML and CSS, I have looked it up on web and moved forward little by little. It took much time because I couldn't notice that one triangle part that I thought it as an image is drawn only CSS. I can draw triangles with only CSS… It is used in the parts of the balloon…

Since I worked hardly about those tasks, The back of my eyes started to hurt in the evening. By the time I finished work, I felt sicked. Therefore I canceled my English lesson and exercising after that. So I took a medicine for eyestrain and slept for about three hours after dinner.

Still feel a little pain, but I feel good now. Although It seems to start Subsurface, I am going to go to bed earlier without watching it.

[dailysoftwareengineerinjapan]

Go out to eat

After COVID-19, we have less chance to go out to eat. But since we are painful to stay home a long time, we went to Ramen shop for a change.

This is a miso-ramen of "Miso-kuma" at Shimbashi in Tokyo. This is very good.

There were mar-oil, fried garlic, grated ginger on each table in "Miso-kuma". Customers are allowed to put them to their ramen as you like. Today, I put fried garlic much and I scooped up them that sank to the bottom of the soup and ate them. Very good !!

However, since I put too much garlic to my ramen, I had to have this soup so much. As a result, I had a slight stomach upset. Arrived at home, I took a stomach medicine immediately...

[dailysoftwareengineerinjapan]

I have been to our office after a long time

Today is my first time to go there this year. Basically we should work from home now, but There were a few people on the floor.

We supposed to start new sprint of scrum from today. So we joined the Sprint Planning.Since It was remote meeting, everyone attended to it from thier home. As a result of dealing with COVID-19, we found that we can fully develop our application even remotely.It is a little difficult to discuss about an issue remotely. But we found that we can fully do that smoothly if we take a time to prepare for it.

After lunch time, I created some JIRA tickets based on the result of Sprint Planning. After that, I moved to the office and do the maintenance work for servers. I was worried that one of them didn't start easily, but all servers have launched finally.

The return train was relatively vacant, but a businessman was still on board. It is hard for people that have tasks that they can't work remotely...

[dailysoftwareengineerinjapan]

Hello, world !

I am a Japanese Software Engineer. I decided to start this English blog as part of studying English. This is a first day!

In Japan, Goverment has decleared a state of emergency for some areas due to the influence of COVID-19 this month. Tokyo, where our office is located, is one of them. Therefore we are working from home now (so-called WFH). So I was at home because it is hard to go out although it is Sunday today.

I want to take the benchmark of the query engine that I am creating now, so I am creating an envrionment for that. The data seems to be about 85 GB. It is 22:50 now but it is not over yet. After that, I am going to convert the data format of each file. When will it end...