Automatic generation of blue calico's single pattern based on Stable Diffusion
Blue calico is a traditional craft printing and dyeing product in China with a long history.It is famous for its distinctive pattern design style and broken lines.However,the lack of an algorithm for the automatic generation of blue calico's single pattern has hindered innovative research on blue calico's patterns.For this reason,an end-to-end automatic generation method of the single blue calico's single pattern was proposed to realize the automatic generation of blue calico's single pattern.Our method is based on a diffusion model,which has been very popular recently.It has achieved great success in the field of image generation,and its main architecture consists of VAE(variational autoencoder),CLIP(contrastive language-image pre-training),and Unet.However,due to the high cost of fine-tuning the entire diffusion model,we choose to use improved DyLoRA(dynamic low-rank adaptation)technology to fine-tune the diffusion model.DyLoRA posits that changes in the parameter matrix during model training cannot achieve full rank.Therefore,the parameter matrix that needs to be updated is transformed into two small matrices multiplied so as to reduce the number of updated parameters.However,this parameter decomposition method has no effect on improving rank,so we improved this technique and proposed a new parameter decomposition method.Through this technology,we can fine-tune the diffusion model at an affordable cost to produce blue calico's single pattern.At the same time,in order to control the generation of blue calico,we also introduced the Controlnet network to control the overall layout of the generated single pattern.There is no objective measurement standard in such experiments,so we used the generated image for visual comparison.In the experiment,to demonstrate the superiority of the proposed algorithm,we compared our algorithm with a model based on the CycleGan algorithm and original DyLoRA.The experimental results show that our proposed algorithm can effectively generate better blue calico single pattern than the other two methods,even though its input is only simple text.In the example,it can be seen that the generated blue calico single pattern conforms to the characteristics of broken lines and connected meanings,and is rich in artistic conception.At the same time,we used the ControlNet network to control the overall structure of the generated single pattern.As a part of national intangible cultural heritage,blue calico has important value and significance in digital inheritance and innovation.This article proposed a method for fine-tuning the diffusion model Stable Diffusion to generate the blue calico's single pattern.This method fully utilized the rich semantic information from the pre-trained Stable Diffusion 1.5 model.Based on this large pre-trained model,the improved DyLoRA fine-tuning method was used to enable the model to learn the style of blue calico's single pattern,and Controlnet was used to limit the structure of the generated content.Finally,we achieved the effect of outputting blue calico's single pattern by inputting appropriate prompt words,and hundreds of sample images were generated according to this method.Next,research will be conducted on the automatic generation of more types and complex blue calico's single pattern.
blue calicodeep learningDyLoRAStable Diffusionsingle patternpattern generation
万方数据
维普
