Daily Papers

An effective reward model plays a pivotal role in reinforcement learning for post-training enhancement of visual generative models. However, current approaches of reward modeling suffer from implementation complexity due to their reliance on extensive human-annotated preference data or meticulously engineered quality dimensions that are often incomplete and engineering-intensive. Inspired by adversarial training in generative adversarial networks (GANs), this paper proposes GAN-RM, an efficient reward modeling framework that eliminates manual preference annotation and explicit quality dimension engineering. Our method trains the reward model through discrimination between a small set of representative, unpaired target samples(denoted as Preference Proxy Data) and model-generated ordinary outputs, requiring only a few hundred target samples. Comprehensive experiments demonstrate our GAN-RM's effectiveness across multiple key applications including test-time scaling implemented as Best-of-N sample filtering, post-training approaches like Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO). Code and data will be released at https://github.com/Visualignment/GAN-RM.

We report the discovery of two broad-line X-ray AGNs cid_414 and cid_947 at z~3 that exhibit prominent He Iλ10830+ Paγ emission and absorption, identified from the JWST Cycle 3 large GO treasury program COSMOS-3D using NIRCam F444W grism spectroscopy. Additional UV/optical line measurements (e.g., Lyα, Si IV, C IV) come from complementary COSMOS-field spectroscopy. Both sources are robustly detected in the mid-infrared, with detections in MIRI F1000W for both AGNs and an additional detection in MIRI F2100W for cid_414, indicating the presence of hot dust emission. The source cid_947 shows a higher He Iλ10830 absorption column density and X-ray-inferred N_{rm H}, and displays strong outflow signatures in He I, Si IV, and C IV with velocity offsets exceeding 5000 km/s. The source cid_414 shows a narrow Lyα emission line with luminosity log L_{rm Lyα}=42.49pm0.01~erg~s^{-1} and a higher intrinsic 2-10 keV X-ray luminosity. Host-galaxy decomposition and multi-component SED fitting indicate that cid_947 hosts a more massive black hole but lower star formation rate than cid_414. From simplified photoionization modeling, we infer that the dense absorbing gas has a characteristic size comparable to the nuclear broad-line region and is likely kinematically coupled to the obscuration associated with the dust torus. He Iλ1083 absorption has also been identified in several compact little red dots at similar redshifts. Together with the two AGNs reported here, these findings suggest that dense circumnuclear gas are plausibly prevalent at high redshift and plays an important role in regulating AGN obscuration and black hole--host co-evolution.

Imperceptible digital watermarking is important in copyright protection, misinformation prevention, and responsible generative AI. We propose TrustMark - a GAN-based watermarking method with novel design in architecture and spatio-spectra losses to balance the trade-off between watermarked image quality with the watermark recovery accuracy. Our model is trained with robustness in mind, withstanding various in- and out-place perturbations on the encoded image. Additionally, we introduce TrustMark-RM - a watermark remover method useful for re-watermarking. Our methods achieve state-of-art performance on 3 benchmarks comprising arbitrary resolution images.