Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Space
🤝 Open to Collab
Junrulu
Junrulu
AI & ML interests
None yet
Recent Activity
upvoted a paper about 1 month ago
From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI authored a paper about 2 months ago
Toward Native Multimodal Modeling: A Roadmap upvoted a paper about 2 months ago
Toward Native Multimodal Modeling: A RoadmapOrganizations
Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence
-
jiazhengli/Pythia-2.8B-HH-RLHF-Iterative-SamPO
Text Generation • 3B • Updated • 4 -
jiazhengli/Pythia-2.8B-TLDR-Iterative-SamPO
Text Generation • 3B • Updated • 5 -
Junrulu/Llama-3-8B-Instruct-Iterative-SamPO
Text Generation • 8B • Updated • 4 • 1 -
Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence
Paper • 2406.10957 • Published • 2
Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Space
A Fine-Grained Composite Benchmark for Role-Playing and Instruction-Following
Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence
-
jiazhengli/Pythia-2.8B-HH-RLHF-Iterative-SamPO
Text Generation • 3B • Updated • 4 -
jiazhengli/Pythia-2.8B-TLDR-Iterative-SamPO
Text Generation • 3B • Updated • 5 -
Junrulu/Llama-3-8B-Instruct-Iterative-SamPO
Text Generation • 8B • Updated • 4 • 1 -
Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence
Paper • 2406.10957 • Published • 2
Tuning LLMs to Use Memos for Consistent Long-Range Open-Domain Conversation