andreasjansson/codellama-34b-instruct-gguf 🔢📝❓ → 📝

▶️ 1.1K runs 📅 Sep 2023 ⚙️ Cog 0.8.2 🔗 GitHub ⚖️ License

code-generation structured-output text-generation

About

CodeLlama-34B-instruct with support for grammars and jsonschema

Example Output

Prompt:

"Write a Python program that counts the number of occurrences of a character in a string."

Output

def num_occurrences(c: str, s: str) -> int:
    count = 0
    for ch in s:
        if c == ch:
            count += 1
    return count

Performance Metrics

5.98s Prediction Time

5.99s Total Time

All Input Parameters

{
  "top_k": 10,
  "top_p": 0.95,
  "prompt": "Write a Python program that counts the number of occurrences of a character in a string.",
  "grammar": "root        ::= \"```python\\ndef num_occurrences(c: str, s: str) -> int:\\n    \" code \"```\"\ncode        ::= [^`()]+",
  "jsonschema": "",
  "max_tokens": 500,
  "temperature": 0.8,
  "mirostat_mode": "Disabled",
  "repeat_penalty": 1.1,
  "mirostat_entropy": 5,
  "presence_penalty": 0,
  "frequency_penalty": 0,
  "mirostat_learning_rate": 0.1
}

Input Parameters

top_k Type: integerDefault: 10: Top K
top_p Type: numberDefault: 0.95: Top P
prompt (required) Type: string: Prompt
grammar Type: string: Grammar in GBNF format. Use either grammar or jsonschema.
jsonschema Type: string: JSON schema for the generated output. Use either grammar or jsonschema.
max_tokens Type: integerDefault: 500: Max number of tokens to return
temperature Type: numberDefault: 0.8: Temperature
mirostat_mode Default: Disabled: Mirostat sampling mode
repeat_penalty Type: numberDefault: 1.1Range: 0 - 2: Repetition penalty
mirostat_entropy Type: numberDefault: 5Range: 0 - 10: Mirostat target entropy
presence_penalty Type: numberDefault: 0Range: 0 - 2: Presence
frequency_penalty Type: numberDefault: 0Range: 0 - 2: Frequency penalty
mirostat_learning_rate Type: numberDefault: 0.1Range: 0 - 1: Mirostat learning rate, if mirostat_mode is not Disabled

Output Schema

Output

Type: array • Items Type: string

Example Execution Logs

from_string grammar:
root ::= [`] [`] [`] [p] [y] [t] [h] [o] [n] [<U+000A>] [d] [e] [f] [ ] [n] [u] [m] [_] [o] [c] [c] [u] [r] [r] [e] [n] [c] [e] [s] [(] [c] [:] [ ] [s] [t] [r] [,] [ ] [s] [:] [ ] [s] [t] [r] [)] [ ] [-] [>] [ ] [i] [n] [t] [:] [<U+000A>] [ ] [ ] [ ] [ ] code [`] [`] [`] 
code ::= code_2
code_2 ::= [^`()] code_2 | [^`()]
Prompt:
Write a Python program that counts the number of occurrences of a character in a string.
Llama.generate: prefix-match hit
llama_print_timings:        load time =   281.28 ms
llama_print_timings:      sample time =   442.12 ms /    56 runs   (    7.90 ms per token,   126.66 tokens per second)
llama_print_timings: prompt eval time =     0.00 ms /     1 tokens (    0.00 ms per token,      inf tokens per second)
llama_print_timings:        eval time =  2299.32 ms /    56 runs   (   41.06 ms per token,    24.36 tokens per second)
llama_print_timings:       total time =  2862.98 ms

Version Details

Version ID: 97a1fb465d5cdf2854c89ebeaee3ceb353206b8187b665a83bcf6efd21e534ab
Version Created: September 22, 2023

Run on Replicate →