nwhitehead/llama2-7b-chat-gptq 🔢📝 → 📝
About
Example Output
Prompt:
"[INST]
Write a short story about a dog that gets lost.[/INST]"
Output
Once upon a time, there was a little dog named Max. Max lived with his family in a cozy house on the outskirts of a small town. He was a playful and curious dog, always eager to explore new places and make new friends.
One day, Max's family decided to go on a camping trip in the nearby woods. Max was so excited to go on an adventure with his family that he could hardly contain himself. He barked and wagged his tail as he followed his family through the woods, sniffing at every tree and rock along the way.
As the day wore on, Max became more and more tired. He lay down under a tree and closed his eyes, feeling the warm sun on his fur. But when he woke up, he was alone. He looked around, but he couldn't see his family anywhere.
Max barked and barked, hoping they would hear him and come back. But there was no answer. He was lost in the woods, all alone.
Max wandered through the woods, sniffing for his family. He followed the scent of their campfire, but it led him in circles. He was lost and scared, and he didn't know what to do.
As the sun began to set, Max curled up under a bush, shivering with fear. He had never been so alone before, and he didn't know how he would find his way back home.
Just as Max was drifting off to sleep, he heard a rustling in the bushes. He barked and growled, ready to defend himself. But then he saw his family, peeking out from behind the bushes. They had come to find him, and they were so happy to see him safe and sound.
Max's family hugged him and stroked his fur, apologizing for losing him. Max wagged his tail and licked their faces, happy to be reunited with his loved ones. From then on, he stuck close to his family, never wanting to be separated from them again.
As they made their way back home, Max couldn't help but think about how lost he had felt before. But he knew that he was lucky to have such a wonderful family, who would always be there to help him find his way.
One day, Max's family decided to go on a camping trip in the nearby woods. Max was so excited to go on an adventure with his family that he could hardly contain himself. He barked and wagged his tail as he followed his family through the woods, sniffing at every tree and rock along the way.
As the day wore on, Max became more and more tired. He lay down under a tree and closed his eyes, feeling the warm sun on his fur. But when he woke up, he was alone. He looked around, but he couldn't see his family anywhere.
Max barked and barked, hoping they would hear him and come back. But there was no answer. He was lost in the woods, all alone.
Max wandered through the woods, sniffing for his family. He followed the scent of their campfire, but it led him in circles. He was lost and scared, and he didn't know what to do.
As the sun began to set, Max curled up under a bush, shivering with fear. He had never been so alone before, and he didn't know how he would find his way back home.
Just as Max was drifting off to sleep, he heard a rustling in the bushes. He barked and growled, ready to defend himself. But then he saw his family, peeking out from behind the bushes. They had come to find him, and they were so happy to see him safe and sound.
Max's family hugged him and stroked his fur, apologizing for losing him. Max wagged his tail and licked their faces, happy to be reunited with his loved ones. From then on, he stuck close to his family, never wanting to be separated from them again.
As they made their way back home, Max couldn't help but think about how lost he had felt before. But he knew that he was lucky to have such a wonderful family, who would always be there to help him find his way.
Performance Metrics
15.17s
Prediction Time
15.15s
Total Time
All Input Parameters
{
"seed": -1,
"top_k": 20,
"top_p": 1,
"prompt": "[INST]\nWrite a short story about a dog that gets lost.[/INST]",
"max_tokens": 1024,
"min_tokens": 1,
"temperature": 0.5,
"repetition_penalty": 1
}
Input Parameters
- seed
- Seed for reproducibility, -1 for random seed
- top_k
- Number of top candidates to keep
- top_p
- Top cumulative probability to filter candidates
- prompt
- Text prompt for the model
- max_tokens
- Maximum tokens to generate
- min_tokens
- Minimum tokens to generate
- temperature
- Temperature of the output, it's best to keep it below 1
- repetition_penalty
- Penalty for repeated tokens in the model's output
Output Schema
Output
Example Execution Logs
Using flash attention 2. Time used: 42ms | Generated: " " Time used: 10ms | Generated: " Once" Time used: 10ms | Generated: " upon" Time used: 10ms | Generated: " a" Time used: 10ms | Generated: " time" Time used: 10ms | Generated: "," Time used: 11ms | Generated: " there" Time used: 10ms | Generated: " was" Time used: 10ms | Generated: " a" Time used: 10ms | Generated: " little" Time used: 14ms | Generated: " dog" Time used: 13ms | Generated: " named" Time used: 10ms | Generated: " Max" Time used: 10ms | Generated: "." Time used: 10ms | Generated: " Max" Time used: 11ms | Generated: " lived" Time used: 10ms | Generated: " with" Time used: 11ms | Generated: " his" Time used: 11ms | Generated: " family" Time used: 10ms | Generated: " in" Time used: 10ms | Generated: " a" Time used: 10ms | Generated: " co" Time used: 10ms | Generated: "zy" Time used: 10ms | Generated: " house" Time used: 12ms | Generated: " on" Time used: 10ms | Generated: " the" Time used: 10ms | Generated: " out" Time used: 10ms | Generated: "sk" Time used: 11ms | Generated: "ir" Time used: 10ms | Generated: "ts" Time used: 10ms | Generated: " of" Time used: 10ms | Generated: " a" Time used: 10ms | Generated: " small" Time used: 10ms | Generated: " town" Time used: 10ms | Generated: "." Time used: 11ms | Generated: " He" Time used: 10ms | Generated: " was" Time used: 10ms | Generated: " a" Time used: 11ms | Generated: " play" Time used: 11ms | Generated: "ful" Time used: 10ms | Generated: " and" Time used: 11ms | Generated: " curious" Time used: 10ms | Generated: " dog" Time used: 10ms | Generated: "," Time used: 10ms | Generated: " always" Time used: 10ms | Generated: " eager" Time used: 10ms | Generated: " to" Time used: 10ms | Generated: " explore" Time used: 10ms | Generated: " new" Time used: 10ms | Generated: " places" Time used: 11ms | Generated: " and" Time used: 10ms | Generated: " make" Time used: 10ms | Generated: " new" Time used: 10ms | Generated: " friends" Time used: 10ms | Generated: "." Time used: 11ms | Generated: " " Time used: 10ms | Generated: "One" Time used: 10ms | Generated: " day" Time used: 11ms | Generated: "," Time used: 10ms | Generated: " Max" Time used: 10ms | Generated: "'" Time used: 10ms | Generated: "s" Time used: 10ms | Generated: " family" Time used: 10ms | Generated: " decided" Time used: 10ms | Generated: " to" Time used: 10ms | Generated: " go" Time used: 10ms | Generated: " on" Time used: 11ms | Generated: " a" Time used: 10ms | Generated: " camp" Time used: 10ms | Generated: "ing" Time used: 11ms | Generated: " trip" Time used: 10ms | Generated: " in" Time used: 10ms | Generated: " the" Time used: 10ms | Generated: " nearby" Time used: 10ms | Generated: " woods" Time used: 10ms | Generated: "." Time used: 11ms | Generated: " Max" Time used: 10ms | Generated: " was" Time used: 11ms | Generated: " so" Time used: 11ms | Generated: " excited" Time used: 10ms | Generated: " to" Time used: 10ms | Generated: " go" Time used: 11ms | Generated: " on" Time used: 11ms | Generated: " an" Time used: 11ms | Generated: " advent" Time used: 11ms | Generated: "ure" Time used: 10ms | Generated: " with" Time used: 10ms | Generated: " his" Time used: 11ms | Generated: " family" Time used: 11ms | Generated: " that" Time used: 11ms | Generated: " he" Time used: 11ms | Generated: " could" Time used: 10ms | Generated: " hardly" Time used: 10ms | Generated: " contain" Time used: 10ms | Generated: " himself" Time used: 11ms | Generated: "." Time used: 11ms | Generated: " He" Time used: 12ms | Generated: " b" Time used: 11ms | Generated: "ark" Time used: 11ms | Generated: "ed" Time used: 11ms | Generated: " and" Time used: 12ms | Generated: " w" Time used: 12ms | Generated: "ag" Time used: 12ms | Generated: "ged" Time used: 12ms | Generated: " his" Time used: 12ms | Generated: " tail" Time used: 12ms | Generated: " as" Time used: 12ms | Generated: " he" Time used: 11ms | Generated: " followed" Time used: 11ms | Generated: " his" Time used: 11ms | Generated: " family" Time used: 12ms | Generated: " through" Time used: 12ms | Generated: " the" Time used: 12ms | Generated: " woods" Time used: 13ms | Generated: "," Time used: 12ms | Generated: " sn" Time used: 12ms | Generated: "iff" Time used: 12ms | Generated: "ing" Time used: 12ms | Generated: " at" Time used: 12ms | Generated: " every" Time used: 12ms | Generated: " tree" Time used: 12ms | Generated: " and" Time used: 12ms | Generated: " rock" Time used: 12ms | Generated: " along" Time used: 12ms | Generated: " the" Time used: 12ms | Generated: " way" Time used: 12ms | Generated: "." Time used: 12ms | Generated: " " Time used: 12ms | Generated: "As" Time used: 12ms | Generated: " the" Time used: 12ms | Generated: " day" Time used: 12ms | Generated: " wore" Time used: 12ms | Generated: " on" Time used: 12ms | Generated: "," Time used: 12ms | Generated: " Max" Time used: 15ms | Generated: " became" Time used: 11ms | Generated: " more" Time used: 11ms | Generated: " and" Time used: 11ms | Generated: " more" Time used: 11ms | Generated: " tired" Time used: 11ms | Generated: "." Time used: 11ms | Generated: " He" Time used: 11ms | Generated: " lay" Time used: 11ms | Generated: " down" Time used: 11ms | Generated: " under" Time used: 11ms | Generated: " a" Time used: 11ms | Generated: " tree" Time used: 11ms | Generated: " and" Time used: 11ms | Generated: " closed" Time used: 11ms | Generated: " his" Time used: 11ms | Generated: " eyes" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " feeling" Time used: 11ms | Generated: " the" Time used: 11ms | Generated: " warm" Time used: 11ms | Generated: " sun" Time used: 11ms | Generated: " on" Time used: 11ms | Generated: " his" Time used: 11ms | Generated: " fur" Time used: 11ms | Generated: "." Time used: 11ms | Generated: " But" Time used: 11ms | Generated: " when" Time used: 11ms | Generated: " he" Time used: 11ms | Generated: " w" Time used: 11ms | Generated: "oke" Time used: 11ms | Generated: " up" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " he" Time used: 11ms | Generated: " was" Time used: 11ms | Generated: " alone" Time used: 11ms | Generated: "." Time used: 11ms | Generated: " He" Time used: 11ms | Generated: " looked" Time used: 11ms | Generated: " around" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " but" Time used: 11ms | Generated: " he" Time used: 11ms | Generated: " couldn" Time used: 11ms | Generated: "'" Time used: 11ms | Generated: "t" Time used: 11ms | Generated: " see" Time used: 15ms | Generated: " his" Time used: 11ms | Generated: " family" Time used: 11ms | Generated: " anywhere" Time used: 11ms | Generated: "." Time used: 11ms | Generated: " " Time used: 11ms | Generated: "Max" Time used: 11ms | Generated: " b" Time used: 11ms | Generated: "ark" Time used: 11ms | Generated: "ed" Time used: 11ms | Generated: " and" Time used: 11ms | Generated: " b" Time used: 11ms | Generated: "ark" Time used: 11ms | Generated: "ed" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " hoping" Time used: 11ms | Generated: " they" Time used: 11ms | Generated: " would" Time used: 11ms | Generated: " hear" Time used: 11ms | Generated: " him" Time used: 11ms | Generated: " and" Time used: 11ms | Generated: " come" Time used: 11ms | Generated: " back" Time used: 11ms | Generated: "." Time used: 11ms | Generated: " But" Time used: 11ms | Generated: " there" Time used: 11ms | Generated: " was" Time used: 11ms | Generated: " no" Time used: 11ms | Generated: " answer" Time used: 11ms | Generated: "." Time used: 11ms | Generated: " He" Time used: 11ms | Generated: " was" Time used: 11ms | Generated: " lost" Time used: 11ms | Generated: " in" Time used: 11ms | Generated: " the" Time used: 11ms | Generated: " woods" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " all" Time used: 11ms | Generated: " alone" Time used: 11ms | Generated: "." Time used: 11ms | Generated: " " Time used: 11ms | Generated: "Max" Time used: 11ms | Generated: " w" Time used: 11ms | Generated: "ander" Time used: 11ms | Generated: "ed" Time used: 11ms | Generated: " through" Time used: 11ms | Generated: " the" Time used: 11ms | Generated: " woods" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " sn" Time used: 11ms | Generated: "iff" Time used: 11ms | Generated: "ing" Time used: 11ms | Generated: " for" Time used: 11ms | Generated: " his" Time used: 11ms | Generated: " family" Time used: 11ms | Generated: "." Time used: 11ms | Generated: " He" Time used: 11ms | Generated: " followed" Time used: 11ms | Generated: " the" Time used: 11ms | Generated: " sc" Time used: 11ms | Generated: "ent" Time used: 11ms | Generated: " of" Time used: 11ms | Generated: " their" Time used: 11ms | Generated: " camp" Time used: 11ms | Generated: "fire" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " but" Time used: 11ms | Generated: " it" Time used: 11ms | Generated: " led" Time used: 11ms | Generated: " him" Time used: 11ms | Generated: " in" Time used: 11ms | Generated: " circles" Time used: 11ms | Generated: "." Time used: 11ms | Generated: " He" Time used: 11ms | Generated: " was" Time used: 11ms | Generated: " lost" Time used: 11ms | Generated: " and" Time used: 11ms | Generated: " sc" Time used: 11ms | Generated: "ared" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " and" Time used: 11ms | Generated: " he" Time used: 11ms | Generated: " didn" Time used: 11ms | Generated: "'" Time used: 11ms | Generated: "t" Time used: 11ms | Generated: " know" Time used: 11ms | Generated: " what" Time used: 11ms | Generated: " to" Time used: 11ms | Generated: " do" Time used: 11ms | Generated: "." Time used: 13ms | Generated: " " Time used: 14ms | Generated: "As" Time used: 11ms | Generated: " the" Time used: 11ms | Generated: " sun" Time used: 11ms | Generated: " began" Time used: 11ms | Generated: " to" Time used: 11ms | Generated: " set" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " Max" Time used: 11ms | Generated: " cur" Time used: 11ms | Generated: "led" Time used: 11ms | Generated: " up" Time used: 11ms | Generated: " under" Time used: 11ms | Generated: " a" Time used: 11ms | Generated: " bush" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " sh" Time used: 11ms | Generated: "iver" Time used: 11ms | Generated: "ing" Time used: 11ms | Generated: " with" Time used: 11ms | Generated: " fear" Time used: 11ms | Generated: "." Time used: 11ms | Generated: " He" Time used: 11ms | Generated: " had" Time used: 11ms | Generated: " never" Time used: 11ms | Generated: " been" Time used: 11ms | Generated: " so" Time used: 11ms | Generated: " alone" Time used: 11ms | Generated: " before" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " and" Time used: 11ms | Generated: " he" Time used: 11ms | Generated: " didn" Time used: 11ms | Generated: "'" Time used: 11ms | Generated: "t" Time used: 11ms | Generated: " know" Time used: 11ms | Generated: " how" Time used: 11ms | Generated: " he" Time used: 11ms | Generated: " would" Time used: 11ms | Generated: " find" Time used: 11ms | Generated: " his" Time used: 11ms | Generated: " way" Time used: 11ms | Generated: " back" Time used: 11ms | Generated: " home" Time used: 11ms | Generated: "." Time used: 11ms | Generated: " " Time used: 11ms | Generated: "Just" Time used: 11ms | Generated: " as" Time used: 11ms | Generated: " Max" Time used: 11ms | Generated: " was" Time used: 11ms | Generated: " dr" Time used: 11ms | Generated: "ifting" Time used: 11ms | Generated: " off" Time used: 11ms | Generated: " to" Time used: 11ms | Generated: " sleep" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " he" Time used: 11ms | Generated: " heard" Time used: 11ms | Generated: " a" Time used: 11ms | Generated: " rust" Time used: 11ms | Generated: "ling" Time used: 11ms | Generated: " in" Time used: 11ms | Generated: " the" Time used: 11ms | Generated: " bush" Time used: 11ms | Generated: "es" Time used: 11ms | Generated: "." Time used: 11ms | Generated: " He" Time used: 11ms | Generated: " b" Time used: 11ms | Generated: "ark" Time used: 11ms | Generated: "ed" Time used: 11ms | Generated: " and" Time used: 11ms | Generated: " grow" Time used: 11ms | Generated: "led" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " ready" Time used: 11ms | Generated: " to" Time used: 11ms | Generated: " defend" Time used: 11ms | Generated: " himself" Time used: 11ms | Generated: "." Time used: 11ms | Generated: " But" Time used: 11ms | Generated: " then" Time used: 11ms | Generated: " he" Time used: 11ms | Generated: " saw" Time used: 11ms | Generated: " his" Time used: 11ms | Generated: " family" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " pe" Time used: 11ms | Generated: "ek" Time used: 11ms | Generated: "ing" Time used: 11ms | Generated: " out" Time used: 11ms | Generated: " from" Time used: 12ms | Generated: " behind" Time used: 12ms | Generated: " the" Time used: 12ms | Generated: " bush" Time used: 13ms | Generated: "es" Time used: 13ms | Generated: "." Time used: 12ms | Generated: " They" Time used: 12ms | Generated: " had" Time used: 11ms | Generated: " come" Time used: 11ms | Generated: " to" Time used: 11ms | Generated: " find" Time used: 11ms | Generated: " him" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " and" Time used: 12ms | Generated: " they" Time used: 11ms | Generated: " were" Time used: 11ms | Generated: " so" Time used: 11ms | Generated: " happy" Time used: 11ms | Generated: " to" Time used: 11ms | Generated: " see" Time used: 11ms | Generated: " him" Time used: 11ms | Generated: " safe" Time used: 11ms | Generated: " and" Time used: 11ms | Generated: " sound" Time used: 11ms | Generated: "." Time used: 11ms | Generated: " " Time used: 11ms | Generated: "Max" Time used: 11ms | Generated: "'" Time used: 11ms | Generated: "s" Time used: 11ms | Generated: " family" Time used: 11ms | Generated: " h" Time used: 11ms | Generated: "ug" Time used: 11ms | Generated: "ged" Time used: 11ms | Generated: " him" Time used: 11ms | Generated: " and" Time used: 11ms | Generated: " st" Time used: 11ms | Generated: "rok" Time used: 11ms | Generated: "ed" Time used: 11ms | Generated: " his" Time used: 11ms | Generated: " fur" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " apolog" Time used: 11ms | Generated: "izing" Time used: 11ms | Generated: " for" Time used: 11ms | Generated: " losing" Time used: 11ms | Generated: " him" Time used: 11ms | Generated: "." Time used: 11ms | Generated: " Max" Time used: 11ms | Generated: " w" Time used: 11ms | Generated: "ag" Time used: 11ms | Generated: "ged" Time used: 12ms | Generated: " his" Time used: 11ms | Generated: " tail" Time used: 11ms | Generated: " and" Time used: 11ms | Generated: " l" Time used: 12ms | Generated: "icked" Time used: 12ms | Generated: " their" Time used: 11ms | Generated: " faces" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " happy" Time used: 11ms | Generated: " to" Time used: 11ms | Generated: " be" Time used: 11ms | Generated: " reun" Time used: 12ms | Generated: "ited" Time used: 12ms | Generated: " with" Time used: 12ms | Generated: " his" Time used: 11ms | Generated: " loved" Time used: 11ms | Generated: " ones" Time used: 11ms | Generated: "." Time used: 11ms | Generated: " From" Time used: 11ms | Generated: " then" Time used: 11ms | Generated: " on" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " he" Time used: 11ms | Generated: " stuck" Time used: 11ms | Generated: " close" Time used: 11ms | Generated: " to" Time used: 11ms | Generated: " his" Time used: 11ms | Generated: " family" Time used: 12ms | Generated: "," Time used: 11ms | Generated: " never" Time used: 11ms | Generated: " wanting" Time used: 11ms | Generated: " to" Time used: 14ms | Generated: " be" Time used: 13ms | Generated: " separated" Time used: 13ms | Generated: " from" Time used: 13ms | Generated: " them" Time used: 14ms | Generated: " again" Time used: 14ms | Generated: "." Time used: 14ms | Generated: " " Time used: 14ms | Generated: "As" Time used: 14ms | Generated: " they" Time used: 13ms | Generated: " made" Time used: 13ms | Generated: " their" Time used: 13ms | Generated: " way" Time used: 14ms | Generated: " back" Time used: 14ms | Generated: " home" Time used: 13ms | Generated: "," Time used: 14ms | Generated: " Max" Time used: 14ms | Generated: " couldn" Time used: 13ms | Generated: "'" Time used: 14ms | Generated: "t" Time used: 14ms | Generated: " help" Time used: 14ms | Generated: " but" Time used: 14ms | Generated: " think" Time used: 14ms | Generated: " about" Time used: 14ms | Generated: " how" Time used: 14ms | Generated: " lost" Time used: 14ms | Generated: " he" Time used: 14ms | Generated: " had" Time used: 14ms | Generated: " felt" Time used: 14ms | Generated: " before" Time used: 13ms | Generated: "." Time used: 13ms | Generated: " But" Time used: 14ms | Generated: " he" Time used: 12ms | Generated: " knew" Time used: 11ms | Generated: " that" Time used: 11ms | Generated: " he" Time used: 11ms | Generated: " was" Time used: 11ms | Generated: " luck" Time used: 11ms | Generated: "y" Time used: 11ms | Generated: " to" Time used: 11ms | Generated: " have" Time used: 12ms | Generated: " such" Time used: 11ms | Generated: " a" Time used: 11ms | Generated: " wonderful" Time used: 11ms | Generated: " family" Time used: 11ms | Generated: "," Time used: 11ms | Generated: " who" Time used: 11ms | Generated: " would" Time used: 11ms | Generated: " always" Time used: 11ms | Generated: " be" Time used: 11ms | Generated: " there" Time used: 11ms | Generated: " to" Time used: 11ms | Generated: " help" Time used: 11ms | Generated: " him" Time used: 11ms | Generated: " find" Time used: 11ms | Generated: " his" Time used: 11ms | Generated: " way" Time used: 11ms | Generated: "." Generated 500 tokens in 5.91241 seconds(84.56792 TPS). (RAM(MB): 64485 / 773879 | VRAM(MB): 5737 / 48548)
Version Details
- Version ID
8c1f632f7a9df740bfbe8f6b35e491ddfe5c43a79b43f062f719ccbe03772b52- Version Created
- September 16, 2023