uwulewd/airoboros-llama-2-70b 🔢📝✓ → 📝

▶️ 17.6K runs 📅 Jul 2023 ⚙️ Cog 0.8.6 🔗 GitHub ⚖️ License

code-generation question-answering text-generation text-translation

About

Inference Airoboros L2 70B 2.1 - GPTQ using ExLlama.

Example Output

Prompt:

"USER: Hello, please code a python function that calculates cosine similarity between 2 vectors, remember to do error checks.
ASSISTANT:"

Output

Here is a Python function that calculates the cosine similarity between two vectors:

def cosine_similarity(vec1, vec2):
    if len(vec1) != len(vec2):
        raise ValueError("Vectors must have the same length")

    dot_product = sum(a * b for a, b in zip(vec1, vec2))
    mag1 = sum(a**2 for a in vec1)
    mag2 = sum(b**2 for b in vec2)
    norm_product = math.sqrt(mag1 * mag2)

    if norm_product == 0:
        return 0

    return dot_product / norm_product

This function takes two vectors as input and checks if they are of the same length. If not, it raises a ValueError exception. It then calculates the dot product, magnitude of each vector, and the norm product. Finally, it returns the cosine similarity between the two vectors.

Performance Metrics

31.85s Prediction Time

31.85s Total Time

All Input Parameters

{
  "seed": -1,
  "top_k": 20,
  "top_p": 1,
  "prompt": "USER: Hello, please code a python function that calculates cosine similarity between 2 vectors, remember to do error checks.\nASSISTANT:",
  "use_lora": false,
  "max_tokens": 1024,
  "min_tokens": 1,
  "temperature": 0.5,
  "repetition_penalty": 1
}

Input Parameters

seed Type: integerDefault: -1Range: -2147483648 - 2147483647: Seed for reproducibility, -1 for random seed
top_k Type: integerDefault: 20Range: 1 - 100: Number of top candidates to keep
top_p Type: numberDefault: 1Range: 0.01 - 1: Top cumulative probability to filter candidates
prompt Type: stringDefault: USER: Hello, who are you? ASSISTANT:: Text prompt for the model
use_lora Type: booleanDefault: false: Whether to use LoRA for prediction
max_tokens Type: integerDefault: 1024Range: 1 - 4096: Maximum tokens to generate
min_tokens Type: integerDefault: 1Range: 0 - 4096: Minimum tokens to generate
temperature Type: numberDefault: 0.5Range: 0.01 - 2: Temperature of the output, it's best to keep it below 1
repetition_penalty Type: numberDefault: 1Range: 1 - 1.5: Penalty for repeated tokens in the model's output

Output Schema

Output

Type: array • Items Type: string

Example Execution Logs

Using flash attention 2.
Time used: 82ms | Generated: " Here"
Time used: 81ms | Generated: " is"
Time used: 81ms | Generated: " a"
Time used: 81ms | Generated: " Python"
Time used: 81ms | Generated: " function"
Time used: 81ms | Generated: " that"
Time used: 81ms | Generated: " calcul"
Time used: 81ms | Generated: "ates"
Time used: 81ms | Generated: " the"
Time used: 81ms | Generated: " cos"
Time used: 81ms | Generated: "ine"
Time used: 81ms | Generated: " similarity"
Time used: 81ms | Generated: " between"
Time used: 81ms | Generated: " two"
Time used: 81ms | Generated: " vectors"
Time used: 81ms | Generated: ":"
Time used: 81ms | Generated: "
"
Time used: 81ms | Generated: "
"
Time used: 81ms | Generated: "```"
Time used: 81ms | Generated: "python"
Time used: 81ms | Generated: "
"
Time used: 81ms | Generated: "def"
Time used: 82ms | Generated: " cos"
Time used: 81ms | Generated: "ine"
Time used: 81ms | Generated: "_"
Time used: 81ms | Generated: "similar"
Time used: 81ms | Generated: "ity"
Time used: 81ms | Generated: "("
Time used: 81ms | Generated: "vec"
Time used: 81ms | Generated: "1"
Time used: 81ms | Generated: ","
Time used: 81ms | Generated: " vec"
Time used: 81ms | Generated: "2"
Time used: 81ms | Generated: "):"
Time used: 82ms | Generated: "
"
Time used: 81ms | Generated: "   "
Time used: 81ms | Generated: " if"
Time used: 81ms | Generated: " len"
Time used: 81ms | Generated: "("
Time used: 81ms | Generated: "vec"
Time used: 81ms | Generated: "1"
Time used: 81ms | Generated: ")"
Time used: 81ms | Generated: " !="
Time used: 81ms | Generated: " len"
Time used: 82ms | Generated: "("
Time used: 81ms | Generated: "vec"
Time used: 81ms | Generated: "2"
Time used: 81ms | Generated: "):"
Time used: 81ms | Generated: "
"
Time used: 82ms | Generated: "       "
Time used: 82ms | Generated: " raise"
Time used: 81ms | Generated: " Value"
Time used: 81ms | Generated: "Error"
Time used: 81ms | Generated: "(""
Time used: 81ms | Generated: "V"
Time used: 81ms | Generated: "ectors"
Time used: 81ms | Generated: " must"
Time used: 81ms | Generated: " have"
Time used: 81ms | Generated: " the"
Time used: 81ms | Generated: " same"
Time used: 81ms | Generated: " length"
Time used: 81ms | Generated: "")"
Time used: 81ms | Generated: "
"
Time used: 81ms | Generated: "
"
Time used: 81ms | Generated: "   "
Time used: 81ms | Generated: " dot"
Time used: 81ms | Generated: "_"
Time used: 81ms | Generated: "product"
Time used: 82ms | Generated: " ="
Time used: 82ms | Generated: " sum"
Time used: 81ms | Generated: "("
Time used: 81ms | Generated: "a"
Time used: 81ms | Generated: " *"
Time used: 81ms | Generated: " b"
Time used: 82ms | Generated: " for"
Time used: 82ms | Generated: " a"
Time used: 81ms | Generated: ","
Time used: 81ms | Generated: " b"
Time used: 81ms | Generated: " in"
Time used: 81ms | Generated: " zip"
Time used: 82ms | Generated: "("
Time used: 81ms | Generated: "vec"
Time used: 81ms | Generated: "1"
Time used: 81ms | Generated: ","
Time used: 81ms | Generated: " vec"
Time used: 81ms | Generated: "2"
Time used: 81ms | Generated: "))"
Time used: 82ms | Generated: "
"
Time used: 81ms | Generated: "   "
Time used: 81ms | Generated: " mag"
Time used: 81ms | Generated: "1"
Time used: 81ms | Generated: " ="
Time used: 82ms | Generated: " sum"
Time used: 81ms | Generated: "("
Time used: 81ms | Generated: "a"
Time used: 81ms | Generated: "**"
Time used: 81ms | Generated: "2"
Time used: 82ms | Generated: " for"
Time used: 82ms | Generated: " a"
Time used: 82ms | Generated: " in"
Time used: 82ms | Generated: " vec"
Time used: 82ms | Generated: "1"
Time used: 82ms | Generated: ")"
Time used: 82ms | Generated: "
"
Time used: 82ms | Generated: "   "
Time used: 82ms | Generated: " mag"
Time used: 82ms | Generated: "2"
Time used: 82ms | Generated: " ="
Time used: 82ms | Generated: " sum"
Time used: 82ms | Generated: "("
Time used: 82ms | Generated: "b"
Time used: 82ms | Generated: "**"
Time used: 82ms | Generated: "2"
Time used: 82ms | Generated: " for"
Time used: 82ms | Generated: " b"
Time used: 82ms | Generated: " in"
Time used: 82ms | Generated: " vec"
Time used: 82ms | Generated: "2"
Time used: 82ms | Generated: ")"
Time used: 82ms | Generated: "
"
Time used: 82ms | Generated: "   "
Time used: 82ms | Generated: " norm"
Time used: 82ms | Generated: "_"
Time used: 82ms | Generated: "product"
Time used: 82ms | Generated: " ="
Time used: 82ms | Generated: " math"
Time used: 82ms | Generated: "."
Time used: 82ms | Generated: "sqrt"
Time used: 82ms | Generated: "("
Time used: 82ms | Generated: "mag"
Time used: 82ms | Generated: "1"
Time used: 82ms | Generated: " *"
Time used: 82ms | Generated: " mag"
Time used: 82ms | Generated: "2"
Time used: 82ms | Generated: ")"
Time used: 82ms | Generated: "
"
Time used: 82ms | Generated: "
"
Time used: 82ms | Generated: "   "
Time used: 82ms | Generated: " if"
Time used: 82ms | Generated: " norm"
Time used: 82ms | Generated: "_"
Time used: 82ms | Generated: "product"
Time used: 82ms | Generated: " =="
Time used: 82ms | Generated: " "
Time used: 82ms | Generated: "0"
Time used: 81ms | Generated: ":"
Time used: 82ms | Generated: "
"
Time used: 82ms | Generated: "       "
Time used: 82ms | Generated: " return"
Time used: 82ms | Generated: " "
Time used: 82ms | Generated: "0"
Time used: 82ms | Generated: "
"
Time used: 82ms | Generated: "
"
Time used: 82ms | Generated: "   "
Time used: 82ms | Generated: " return"
Time used: 82ms | Generated: " dot"
Time used: 82ms | Generated: "_"
Time used: 82ms | Generated: "product"
Time used: 82ms | Generated: " /"
Time used: 82ms | Generated: " norm"
Time used: 82ms | Generated: "_"
Time used: 82ms | Generated: "product"
Time used: 82ms | Generated: "
"
Time used: 82ms | Generated: "```"
Time used: 82ms | Generated: "
"
Time used: 82ms | Generated: "
"
Time used: 82ms | Generated: "This"
Time used: 82ms | Generated: " function"
Time used: 82ms | Generated: " takes"
Time used: 82ms | Generated: " two"
Time used: 82ms | Generated: " vectors"
Time used: 82ms | Generated: " as"
Time used: 82ms | Generated: " input"
Time used: 82ms | Generated: " and"
Time used: 82ms | Generated: " checks"
Time used: 82ms | Generated: " if"
Time used: 82ms | Generated: " they"
Time used: 82ms | Generated: " are"
Time used: 82ms | Generated: " of"
Time used: 82ms | Generated: " the"
Time used: 82ms | Generated: " same"
Time used: 82ms | Generated: " length"
Time used: 82ms | Generated: "."
Time used: 82ms | Generated: " If"
Time used: 82ms | Generated: " not"
Time used: 82ms | Generated: ","
Time used: 82ms | Generated: " it"
Time used: 82ms | Generated: " ra"
Time used: 82ms | Generated: "ises"
Time used: 82ms | Generated: " a"
Time used: 82ms | Generated: " Value"
Time used: 82ms | Generated: "Error"
Time used: 82ms | Generated: " exception"
Time used: 82ms | Generated: "."
Time used: 82ms | Generated: " It"
Time used: 82ms | Generated: " then"
Time used: 82ms | Generated: " calcul"
Time used: 82ms | Generated: "ates"
Time used: 82ms | Generated: " the"
Time used: 82ms | Generated: " dot"
Time used: 82ms | Generated: " product"
Time used: 82ms | Generated: ","
Time used: 82ms | Generated: " magnitude"
Time used: 82ms | Generated: " of"
Time used: 82ms | Generated: " each"
Time used: 82ms | Generated: " vector"
Time used: 82ms | Generated: ","
Time used: 82ms | Generated: " and"
Time used: 82ms | Generated: " the"
Time used: 82ms | Generated: " norm"
Time used: 82ms | Generated: " product"
Time used: 82ms | Generated: "."
Time used: 82ms | Generated: " Finally"
Time used: 82ms | Generated: ","
Time used: 82ms | Generated: " it"
Time used: 82ms | Generated: " returns"
Time used: 82ms | Generated: " the"
Time used: 82ms | Generated: " cos"
Time used: 82ms | Generated: "ine"
Time used: 82ms | Generated: " similarity"
Time used: 82ms | Generated: " between"
Time used: 82ms | Generated: " the"
Time used: 82ms | Generated: " two"
Time used: 82ms | Generated: " vectors"
Time used: 82ms | Generated: "."
Time used: 82ms | Generated: "
"
Generated 226 tokens in 18.66739 seconds(12.10668 TPS).
(RAM(MB): 49852 / 1031704 | VRAM(MB): 41865 / 81052)

Version Details

Version ID: ae090a64e6b4468d7fa85c6ca33c979b3cd941c12b1cfa2a237b4a7aa6ebaac4
Version Created: September 5, 2023

Run on Replicate →