obsxrver/ComfyUI-Native-INT8_ConvRot

ComfyUI-Native-Int8-ConvRot

Int8 ConvRot models, converted to the format ComfyUI Expects.

Int8 ConvRot is the best quantization method so far in terms of Quality:Performance ratio. In my personal experience Int8 ConvRot models provide a similar level of quality to BF16 at a generation time matching or beating FP8_Scaled

'INT8 ConvRot is row-wise INT8 with parameters and activations rotated before quantization via ConvRot.' https://github.com/BobJohnson24/ComfyUI-INT8-Fast/blob/main/Metrics.md

Quality Ranking:

GGUF Q8 > INT8 ConvRot > MXFP8 > FP8 >= INT8 Row > INT8 Tensorwise

To Quantize a model to int8;

   pip install -U convert-to-quant --no-deps --force-reinstall --no-cache-dir

   ctq -i source_model_bf16.safetensors -o converted_model_int8_convrot.safetensors \
   --int8 --scaling_mode row --simple --convrot --convrot-group-size 256 \
   --comfy_quant --save-quant-metadata --<model arch flag>

Notes:

1. --convrot-group-size should be 64, 256, or 1024. 256 is generally recommended.
2. --model arch flag is a specific preset depending on the architecture of the model you are trying to quantize. eg --wan --zimage... use ctq --help-filters for the full list.

References:

1. https://www.reddit.com/r/StableDiffusion/comments/1uimp1j/so_is_int8convrot_the_new_hot_thing/
2. https://github.com/BobJohnson24/ComfyUI-INT8-Fast/blob/main/Metrics.md
3. https://huggingface.co/Comfy-Org/Boogu-Image/discussions/10#6a404ed359b6d5b4e834a644
4. https://github.com/Comfy-Org/ComfyUI/pull/14636
5. https://huggingface.co/bertbobson/ComfyUI-INT8_ConvRot
6. https://github.com/silveroxides/convert_to_quant