htao-des

[htao-des]

Design Tech Capstone Week 11

This week the main focus is on filming and video editing as well as some logo design for implementation phase. The filming were done according to the storyboard, but due to my lack of experience in filming, the clips do not seem high quality. For three days there were no progress due to this bottleneck. So I decided to move on despite the lack of quality, after all some defects can be fixed by video editing and filters. But my video editing skills are also lack luster, after hours of trial and error, I finally decide to ask a technician for advice on how to video edit. For future projects, instead of getting stuck on a project for a long time, I would ask someone with experience for assistance.

In terms of logo design, due to my lack of graphic design skills, I decide to use an AI to help with the progress, experiencing the technology first hand. I first ask ChatGPT to create a prompt then the prompt is send to stable diffusion where logo were created for inspiration. These will be redrawn to ensure there are no copyright and graphical issues, as AI can be unpredictable at times and may create something unexpected. Currently, the none of the logo were at a satisfiable level, so more prompt engineering is needed.

[htao-des]

Week 10 des tech capstone

This week is on refining the UI and aesthetic of the prototype. I have decided that a fully functional prototype will not be possible, so for the showcase the functions prototype will be displayed separately. Currently there are three functions that can be showcase.

A prototype for showcasing the Chat function with the AI, this would be on a laptop as a unity game.

Another prototype would showcase the Figma prototype display on the phone.

The last prototype would be a hologram made from transparent plastic and a digital displays.

[htao-des]

Week 9 Design tech capstone

This week the focus is on adding new features to the app. The main achievement being the addition of a camera system for AR and additional work on the goal system for the inworld AI, along with some UI and aesthetic changes.

Here the avatar was "placed" in real life through the iphone camera, this is a test prototype, some minor adjustment is necessary such as the aspect ratio. But the prototype when being tested is experiencing extreme lag, this aspect of the prototype will probably need to be display through video or other forms as live will not function.

In the image there is also the new avatar for the AI. Previous comment on the avatar includes: not enough clothe and the character feel too out of place. Due to the experimental nature of the previous character, many parts were kept minimal (ie less polygon, less bones/ moving parts, more simple color mapping). So the new avatar now has more coverage.

[htao-des]

Week 8 Design Tech Capstone

The focus of this week is the preparation for video production which is part of the "create a pitch" step with the IDEO design kit. This would facilitate the process of video production and to prepare props or collect/ create assets.

Story structure

Describe the context: 20s

Depression amongst young Aucklanders (social isolation, changes in life, stress from work, bad relationships, lack of stress relieve)

Your aim: 20s

Kickstart the recovery journey of people who is lost and do not know how to begin

What you did: 20s

researches and stats on depression, purpose of the app

How will it work: 20s

showcase functionality, show the protagonist recovering

Summary: 20s

problem--> this solution is meant to be temporary and help user find a real solution

Next steps: 20s

scalability, app development, collaboration

Credits – Crew members, course, year and uni logo in a single page

The structure originates from lecture slide and is strongly advised to adhere to. As there are 6 sections within 2 minutes, each section will be 20 seconds long, for some parts this may be easier, but will definitely be very short on time for most. To counteract this, each section can have +/- 5 seconds so more time can be distributed to more important parts. This will be my first video production with editing and assets to help pitch an idea, but the production will be bottlenecked by prototyping process which is iterative and will take a long time and the final high fidelity prototype is needed for the production. Time management will be of essence in here.

Pre-production

Concept:

The recovery journey of an emotionally lost boy following the advice from the APP, the features of the app allow the boy to converse with a kind and caring AI which provide him with valuable advices and set goals for the boy to re enter society.

Logline:

A boy is emotionally lost from being socially isolated, when he is at his lowest point, he discovers the APP and help motivate him to get back on his feet.

Script:

Depression is a complex and multi faceted issue with a large range of factor that contributes to the issue and is different from person to person. One of the main cause of depression is the sudden changes in life and a lack of social interaction. Presence of depression can negatively impact a person’s performance and cognition leading to self-doubt which feedback to depression. In The recent years, lockdown has forced many to stay homebound for a few years, by staying homebound, one may not be able to interact with others and make meaningful connection. The feeling of uncertainties and anxiety due to recent events accompany by loneliness would wear away at the person’s mental wellbeing. The weakened mental health may cause one to become more reclusive, not allowing them to form meaningful connection, this eventually forms a negative feedback loop leading to depression. Without an outlet to vent and manage their emotion, one may find themselves trap in this loop.

The research will be used to design an outlet to guides the users emotion through this ever changing world with use of emerging technologies.The platform would use a personalised AI, for forming personalised solution for everyone to escape the loop, The AI can also act as a support and help their stress tolerance. This research would allow more people to be more sociable to making connections with each other and decreasing the amount of people who are left out and alone.

Storyboard:

Shotlist

show his incompetency--> boy depress in corner --> look at phone (app) --> AI first converse with boy --> show the boy's goal --> AI set goals for him --> AI motivating him to complete goals --> over time the boy no longer need reminders --> show his improvement

Recce

Allocate roles

Collect assets - props, images, music

Here is the pre production template, the script is not finalized. The shotlist took the longest to decide as all these ideas need to fit within a 2 minute video while explaining the function of the APP along with stating the problem. Working on pre production allow me to visualize the general feel and flow of the video and what should be included within the 2 minute time. The process took longer than expected, as this is a major part of assignment 3 more time will be invested here.

[htao-des]

Week 7 Design capstone tech

This week will be mainly for reflections and writings for the report, the two weeks of Unity learning experience allow me to make some progress upon the prototype, but there are still a lot more that can be done, for example the entire process could be shortened significantly if I was able to learn the InworldAI SDK before I began. This is due to the modularity of the SDK allowing a large variety of customization. In the future, I should approach the methodology by exploring all possibility beforehand.

For Assignment 3 a 2 minute video is required, the video will follow the structure of problem - solution - explanation - further steps. Each section should have around 30 seconds, to showcase a point. The video will work as a summary of the functionality for the prototype and is a major part of the assignment so preparation is essential. The current plan is to film using mobile phone and the scene would take place mostly indoors, the script and the composition is still being planned bust should roughly follow the structure of the storyboard.

[htao-des]

Design capstone mid sem 2

This week is the second week of the semester break, this week focuses mainly on app development and AI integration. The Inworld AI provides a guide on integration process in unity. I created an empty scene to simplify the prototype, and made an inworld AI with desirable qualities. The Inworld AI uses the ready player me avatar by default, but as the feedback show a relative disinterest in the style, a new avatar is needed. So for now the avatar will be empty or have a placeholder.

The Inworld AI as well as the controllers were implemented into Unity, with a spare 3D model was used as placeholder for the avatar, here is the result:

The UI is simplified, the UI from Figma will be added along with a background in the future. The aspect ratio will also be changed into 1080 x 1920 for mobile device. More emotions are needed along with expressions to convey more feelings, there is also the possibility of changing the voice using a different engine. These changes will all depend on user feedback.

[htao-des]

Tech capstone midsem break 1

This week, a survey was created to decide on what type of virtual character should be made for the high fidelity prototype. Currently, the response was that a human character in 3D cartoon style is preferred, the reason given was that human character allows the user to form emotionally connection easier due to empathy. As there are only 2 response, the sample size is not big enough to decide. So the priority will be on UI layout and app development.

For app development, I began on figma due to its ability to create a prototype of an app with relative ease. The plan is that the main feature of the app (chatting with the AI) would be made in Unity (due to the ability to integrate the Inworld API). The current Figma prototype requires feedback for iteration, meanwhile the Unity prototype would need a basic UI.

[htao-des]

Week 6 Tech Capstone

This week the main focus is on prototyping and to acquire feedback from classmates. Prototyping includes wireframing, sketching and character design for the virtual character. These prototypes would allow me to gain feedback from classmates so I can work on the high fidelity prototype over the mid semester break.

From the secondary research, an app would be the better option for this capstone due to feasibility and accessibility. To create an app, the basic wireframe has to be mapped out first.

The app layout is heavily inspired by Instagram and Discord. The first line is the registration page and the landing page, for new users to sign up. The second line involves all the functions within the home page and the third line is the AI chat function. This helped me visualize the UX/UI of the app, but further feedback is required to iterate upon this, personally the page number can be a bit much especially for a prototype to present and may cause the user to be confused.

Next I did some sketching for possible virtual characters, I began by brainstorming possible characters. I eventually settle for hedgehog, plant, chameleon, robot, human.

Hedgehog and chameleon were chosen due to their expressive nature, the hedgehog would erect their spikes in self defense and chameleon change color base on mood. These features can be used in the app to indicate danger or threat, maybe if the user uses too much self deprecation or has a threatening tone, or maybe is simply going through tough times and want to share the sentiment with someone (i.e. feel sad together). The robot idea did not go far due to complexity and their functions can be replace by other characters. The plant idea is that it will grow faster if the user is happy (go outside? no sad tone?) for x amount of days. Human idea is still under progress and would be the most difficult to develop (most difficult to get right due to uncanny valley effect). Overall, feedback is required for iteration.

I have made a start to the survey, the survey began with ethics section and follow by the explanation of the project. All avatars above would be shown along with other preexisting ones, this would allow for comparison between industry standard avatars and more unique avatars, as well as showing the preference of the public. The survey is WIP.

[htao-des]

Week 5 Tech Capstone

This week the focus is mainly on ideation phase of the IDEO human centered design, theory of change, journey mapping and storyboard are completed.

The theory of change framework is based on the impact ladder and the ecosystem mapping from the week before. The theory of change is used for finding what shifts/ impacts of the design is most suited for the problem statement. Working through the TOC, I find that the solution of using an AI chat app would be the most applicable due to the accessibility, simplicity and provides higher priority of impacts. The risk is also clear, due to the app being AI based there are chances of misuse AI, and AI providing bad advice or wrong information. While the TOC provides an insight into the specific changes of the design solution, but the framework does not take into account any other possible changes, maybe other frameworks can be used in tandem to make TOC more accurate.

Next, I worked on the journey map for the user, which helps visualize UX from start to end. This eventually lead to storyboarding.

The UX design is all based on the persona mapping done earlier, but the main problem is that this is all theoretical and not real users, and due to the code of ethics, the only user testing can only be done to classmate in the same course which may not be the perfect fit for the target audience. The storyboarding template is also very limiting and accounts for perfect outcome without the consideration of failure, maybe more storyboards can be done that does not have perfect outcomes.

[htao-des]

Week 4 tech capstone

This week the progress is mostly on the presentation, the presentation needs to contain the updated version of the capstone projects, the slides needs to be within five page.

The slides are done with Microsoft designer to test the layout created AI, while the individual slides are good on their own, the AI does not create a coherent layout. This shows that AI have issue with staying consistent.

Besides the slides, four persona were also created due to the UoA ethics preventing third party to be involve in the project in any way. While my target audience do somewhat overlap with classmate, a large majority does not, so persona were created to account for the audience that my classmate do not cover.

[htao-des]

Week 4 Tech Capstone

This week some work were done following the IDEO design kit, frame your design, impact ladder and ecosystem mapping work sheet were completed.

The literature review from previous week help gave insight into filling this worksheet. From going through the worksheet, I realized that the how might we statement need to be change to better fit the topic. While I did complete this worksheet, I feel like more insight can be inputted into it and the fact that this was done in less time than the recommendation on the website suggests that more work can be done.

Here is the impact ladder (left) and ecosystem mapping (right), both would be used to complete the theory of change framework. While the website provided comprehensive guide for completing the impact ladder, there are very little information on the ecosystem mapping. The work was done on rough estimation of the framework, but these two framework probably will not play a huge role in the final project as these would be edited when working through the theory of change.

Lastly for this week, I worked on the presentation slides and scripts for Monday. Discussion on depression is difficult as it is a multi faceted issue and relating this all back to emerging technology is not an easy task.

[htao-des]

Tech Capstone Week 2

This week is the preliminary research phase of the project, I summarize 10 literature on mental health and chatbots.

From these articles four main points can be seen:

Chatbots can be incorrect, repetitive and would be ineffective for mental health treatment

People may not interact with the AI as intended

Depression can be due to isolation, lack of social interaction, lack of a sufficient support relationship and lack of distress tolerance.

Social media can display a skewed reality to actual world leading to self doubt and lowered self esteem within the user

There are concerns with accuracy of Chatbots and error can become detrimental to people's health, so instead of having an AI psychiatrist to help treat mental health, the AI can act more like a triage or a friend and become the support structure to help people sort out their emotions. Meanwhile these information can help an actual psychiatrist understand the person better, if their symptoms are too complex for chatbot to address.

So this is the new how might we statement:

How might we provide a supportive AI for young men in Auckland to help in stress tolerance, mood regulation and changes in life?

From this statement a rough outline of the project timeline is formed:

Now the current challenge is the prototype and what platform to make it in. There are currently three option: webpage, app and vr, a survey may be required to understand which would be more accessible to users and which they prefer to use. The virtual character is also a problem as there are too many styles and form it can take, multiple low fidelity prototype is required.

Overall, I did not manage to work as much as expected due to my other capstone, hopefully when I get more involved in this project I would progress faster.

[htao-des]

Design tech capstone blog 1

Week 1 of design capstone

This capstone project follows the IDEO design kit to address mental health issues through the use of emerging technologies. The first week would be mainly use for project planning and preliminary researches.

Through the use of brainstorming, a list of emerging technologies were shown. As the design challenge were to address mental health, not all of these tools are applicable, but some that can be considered are AR/VR and AI.

AR/VR can be used to simulate reality, currently there are application of these technologies in training people for operations without real life consequences. In theory this technology can be applied to mental health to help people overcome certain difficulties, but there can also be the possible consequence of addiction or apathy towards real world.

AI have wide application, but different AIs are require for different jobs and needed to be trained separately which require large amount of power. There are also ethical and privacy concerns for AI as the data input comes from unknown users and is often manually filtered which could lead to bias. The algorithm that the AI comes up with can often become too complex leading to unpredictable results. Overall, AI can be a great tool in the right hands.

The current progress is slowed as the connection between emerging technologies and mental health issue is still not strong, maybe the connection would be made stronger in the future as I do more research on the topic and work on the prototype of the capstone. Due to the uncertainties and assignment requirements, I worked on the limitations and scopes to help better understand the project.