[开源]CUGAN_TF——运行在浏览器上的动漫超分工具

JustForFun

介绍
当前最优秀的动漫领域超分模型之一Real-CUGAN的tensorflow实现，依赖tfjs框架完成自适应后端的能运行在浏览器上的动漫超分工具。

原版实现分为切块后超分与整图超分，两种都以实现，但切块超分版本转为tfjs模型后在网页运行不正常，已向tfjs仓库提交issue。目前预览版本是整图超分版本，由于内存限制，限制了原始图片大小（512x512以内），后续issue解决将发布切块超分，大概率将不会有限制。

This module is an implementation of one of the current leading anime super-resolution models in the field, Real-CUGAN, using TensorFlow. It relies on the tfjs framework to create an adaptive backend, enabling it to run as an anime super-resolution tool in web browsers.

The original implementation offers two versions: chunk-based super-resolution and full-image super-resolution, both of which have been implemented. However, the chunk-based super-resolution version encounters issues when converted into a tfjs model and run in a web browser. An issue has been raised in the tfjs repository to address this problem.

The current preview version focuses on full-image super-resolution but has limitations on the original image size (up to 512x512) due to memory constraints. Once the issue is resolved, the chunk-based super-resolution version is likely to be released without such limitations.

预览


效果
输入


无降噪输出

保守输出


降噪输出


pytorch模型迁移到tensorflow应该注意的点
图片处理默认维度顺序： tensorflow为nhwc，pytorch为nchw，卷积权重维度顺序也不相同
tensorflow转置卷积无法自定义padding： Conv2DTranspose层padding设为0，后续使用slice手动crop输出
tf.pad无法接受负数：使用 tf.slice作为替代
多尺寸输入导致无法batch ：无解
延迟设置输入尺寸运行时获取size：这是tensorflow图模型的限制
python操作逻辑最好翻译为tensorflow 分支选择api
TensorArray：TensorArray是图模式中python list替代品， TensorArray在eager模式TensorArray.write(i, x)可以直接生效，而在graph模式时需要将引用赋值给自身

C₆H₁₂O₆

tf感觉很难用啊