Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
valine
on Dec 8, 2023
|
parent
|
context
|
favorite
| on:
Mistral "Mixtral" 8x7B 32k model [magnet]
How much data do you need for UNA? Is a typical fine tuning dataset needed or can you get away with less than that?
brucethemoose2
on Dec 8, 2023
|
next
[–]
In addition to what was said, if its anything like DPO you don't need a
lot
of data, just a good set. For instance, DPO requires "good" and "bad" responses for each given prompt.
fblgit
on Dec 8, 2023
|
prev
[–]
doesn't require much data, in a 7B can take a couple hours ~
valine
on Dec 8, 2023
|
parent
[–]
That’s cool. A couple hours on a single GPU or like 8x a100s?
Consider applying for YC's Fall 2026 batch!
Applications
are open till July 27.
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
