Cloth-net: Automation of Training Data Generation Pipeline for Cloth Recognition


Engineering and Applied Sciences


Assistant Professor Of Computer and Information Science

Project Summary

I worked with Professor Chenfanfu Jiang this past summer in the Computer and Information Science department on a project involving the creation of a synthetic data set including cloth models in indoor scenes and the automation of this data creation pipeline. Before deciding on the final pipeline, I tested out numerous tools for each step of the process, whether it was by evaluating different packages for loading cloth mesh models or writing a low-level OpenGL viewer to implement the rendering and output of data. We eventually decided on a more high-level approach with the popular Unreal Engine with scripting tools and computer vision plug-ins to improve modifiability and replicability.

Throughout my whole experience, I was able to take ownership of my project and influence the direction of my work at each step by discussing my ongoing insights with the post-doc and professor. Through this, I gained a glimpse of the nature of computer science research and aspects of software engineering. Instead of using given tools to implement a known project, I was finding and testing out my own tools, many of which were eventually unused in the path to achieving the goal of the project most efficiently and reliably. I also learned about the importance of open-source software, as its wide-spread availability allowed me to borrow the efforts of a larger community to achieve my goals, instead of having to implement everything by myself. Overall, my research broadened my perspective as I continue my computer science education, and it makes me even more excited about my software projects in the future.