q-future/one-align
zero-shot-image-classificationtransformerstransformerspytorchmplug_owl2feature-extractionzero-shot-image-classificationcustom_code
- License
- mit
- Size
- not mirrored yet
Cataloged from q-future/one-align
The model that corresponds to Q-Align (ICML2024).
Quick Start with AutoModel
For this image, 
start an AutoModel scorer with transformers==4.36.1:
import requests
import torch
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("q-future/one-align", trust_remote_code=True, attn_implementation="eager",
torch_dtype=torch.float16, device_map="auto")
from PIL import Image
url = "https://raw.githubusercontent.com/Q-Future/Q-Align/main/fig/singapore_flyer.jpg"
image = Image.open(requests.get(url,stream=True).raw)
model.score([image], task_="quality", input_="image")
# task_ : quality | aesthetics; # input_: image | video
Result should be 1.911 (in range [1,5], higher is better).
From paper: arxiv.org/abs/2312.17090.
Syllabus
IQA Results (Spearman/Pearson/Kendall)
| Datasets | KonIQ (NR-IQA, seen) | SPAQ (NR-IQA, Seen) | KADID (FR-IQA, Seen) | LIVE-C (NR-IQA, Unseen) | LIVE (FR-IQA, Unseen) | CSIQ (FR-IQA, Unseen) | AGIQA (AIGC, Unseen) |
|---|---|---|---|---|---|---|---|
| Previous SOTA | 0.916/0.928 (MUSIQ, ICCV2021) | 0.922/0.919 (LIQE, CVPR2023) | 0.934/0.937 (CONTRIQUE, TIP2022) | NA | NA | NA | NA |
| Q-Align (IQA) | 0.937/0.945/0.785 | 0.931/0.933/0.763 | 0.934/0.934/0.777 | 0.887/0.896/0.706 | 0.874/0.840/0.682 | 0.845/0.876/0.654 | 0.731/0.791/0.529 |
| Q-Align (IQA+VQA) | 0.944/0.949/0.797 | 0.931/0.934/0.764 | 0.952/0.953/0.809 | 0.892/0.899/0.715 | 0.874/0.846/0.684 | 0.852/0.876/0.663 | 0.739/0.782/0.526 |
| OneAlign (IQA+IAA+VQA) |