andreasjansson/llama-2-13b-gguf 📝🔢 → 📝
About
Llama-2 13B with support for grammars and jsonschema
Example Output
Prompt:
"
Send a email from andreas86@telia.se to myfriend@telia.se where you discuss the weather. In the body, describe the current weather in Stockholm as detailed as possible.
Respond with json that adheres to the following jsonschema:
{jsonschema}
"Output
{"bcc":[],"body":"Hello, today it's rainy and 14 degrees.","cc":[],"from":"andreas86@telia.se","subject":"Today weather in Stockholm","to":["myfriend@telia.se"]}
Performance Metrics
5.12s
Prediction Time
5.14s
Total Time
All Input Parameters
{
"top_k": 10,
"top_p": 0.95,
"prompt": "Send a email from andreas86@telia.se to myfriend@telia.se where you discuss the weather. In the body, describe the current weather in Stockholm as detailed as possible.\n\nRespond with json that adheres to the following jsonschema:\n\n{jsonschema}",
"grammar": "",
"jsonschema": "{\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"properties\": {\n \"from\": {\n \"type\": \"string\",\n \"format\": \"email\",\n \"description\": \"The email address of the sender.\"\n },\n \"to\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\",\n \"format\": \"email\"\n },\n \"description\": \"An array of email addresses to send the email to.\"\n },\n \"cc\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\",\n \"format\": \"email\"\n },\n \"description\": \"An array of email addresses to be included in the CC field.\"\n },\n \"bcc\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\",\n \"format\": \"email\"\n },\n \"description\": \"An array of email addresses to be included in the BCC field.\"\n },\n \"subject\": {\n \"type\": \"string\",\n \"description\": \"The subject of the email.\"\n },\n \"body\": {\n \"type\": \"string\",\n \"description\": \"The body text of the email.\"\n }\n },\n \"required\": [\"from\", \"to\", \"subject\", \"body\"],\n \"additionalProperties\": false\n}",
"max_tokens": 500,
"temperature": 0.8,
"mirostat_mode": "Disabled",
"repeat_penalty": 1.1,
"mirostat_entropy": 5,
"presence_penalty": 0,
"frequency_penalty": 0,
"mirostat_learning_rate": 0.1
}
Input Parameters
- prompt (required)
- Prompt
- grammar (required)
- Grammar in GBNF format
- max_tokens
- Max number of tokens to return
Output Schema
Output
Example Execution Logs
from_string grammar:
space ::= space_1
space_1 ::= [ ] |
string ::= ["] string_5 ["] space
string_3 ::= [^"\] | [\] string_4
string_4 ::= ["\/bfnrt] | [u] [0-9a-fA-F] [0-9a-fA-F] [0-9a-fA-F] [0-9a-fA-F]
string_5 ::= string_3 string_5 |
bcc ::= [[] space bcc_10 []] space
bcc_7 ::= string bcc_9
bcc_8 ::= [,] space string
bcc_9 ::= bcc_8 bcc_9 |
bcc_10 ::= bcc_7 |
cc ::= [[] space cc_15 []] space
cc_12 ::= string cc_14
cc_13 ::= [,] space string
cc_14 ::= cc_13 cc_14 |
cc_15 ::= cc_12 |
to ::= [[] space to_20 []] space
to_17 ::= string to_19
to_18 ::= [,] space string
to_19 ::= to_18 to_19 |
to_20 ::= to_17 |
root ::= [{] space ["] [b] [c] [c] ["] space [:] space bcc [,] space ["] [b] [o] [d] [y] ["] space [:] space string [,] space ["] [c] [c] ["] space [:] space cc [,] space ["] [f] [r] [o] [m] ["] space [:] space string [,] space ["] [s] [u] [b] [j] [e] [c] [t] ["] space [:] space string [,] space ["] [t] [o] ["] space [:] space to [}] space
Prompt:
Send a email from andreas86@telia.se to myfriend@telia.se where you discuss the weather. In the body, describe the current weather in Stockholm as detailed as possible.
Respond with json that adheres to the following jsonschema:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"from": {
"type": "string",
"format": "email",
"description": "The email address of the sender."
},
"to": {
"type": "array",
"items": {
"type": "string",
"format": "email"
},
"description": "An array of email addresses to send the email to."
},
"cc": {
"type": "array",
"items": {
"type": "string",
"format": "email"
},
"description": "An array of email addresses to be included in the CC field."
},
"bcc": {
"type": "array",
"items": {
"type": "string",
"format": "email"
},
"description": "An array of email addresses to be included in the BCC field."
},
"subject": {
"type": "string",
"description": "The subject of the email."
},
"body": {
"type": "string",
"description": "The body text of the email."
}
},
"required": ["from", "to", "subject", "body"],
"additionalProperties": false
}
llama_print_timings: load time = 576.68 ms
llama_print_timings: sample time = 518.20 ms / 63 runs ( 8.23 ms per token, 121.57 tokens per second)
llama_print_timings: prompt eval time = 576.15 ms / 411 tokens ( 1.40 ms per token, 713.36 tokens per second)
llama_print_timings: eval time = 1207.52 ms / 62 runs ( 19.48 ms per token, 51.34 tokens per second)
llama_print_timings: total time = 2441.55 ms
Version Details
- Version ID
b988dba0de3af274d9d913eaf277debab37602251b8315eccbfddf9c29d82983- Version Created
- September 19, 2023