lucataco/mistrallite 📝🔢 → 📝

▶️ 663 runs 📅 Oct 2023 ⚙️ Cog 0.8.5 🔗 GitHub ⚖️ License
long-context text-generation

About

MistralLiteA is a fine-tuned Mistral-7B-v0.1 language model, with enhanced capabilities of processing long context (up to 32K tokens)

Example Output

Prompt:

"What are the main challenges to support a long context for LLM?"

Output

The main challenges to support a long context LLM include:

  1. Data availability: A long context LLM requires a large amount of data to train on, which can be difficult and expensive to collect and annotate.

  2. Computational resources: Training a long context LLM requires a lot of computational resources, including powerful hardware and software, which can be costly and difficult to obtain.

  3. Model complexity: A long context LLM is likely to be a complex model, which can be difficult to train and optimize.

  4. Evaluation: Evaluating the performance of a long context LLM can be challenging, as it may not be clear what metrics to use or how to interpret the results.

  5. Human evaluation: A long context LLM may produce outputs that are difficult for humans to understand or interpret, which can make it difficult to evaluate the model's performance.

  6. Ethical considerations: A long context LLM may raise ethical concerns, such as the potential for bias or the impact on privacy and security.

Performance Metrics

10.15s Prediction Time
12.29s Total Time
All Input Parameters 
{
  "prompt": "What are the main challenges to support a long context for LLM?",
  "max_new_tokens": 400
}
Input Parameters
prompt Type: stringDefault: What are the main challenges to support a long context for LLM?
Input prompt
max_new_tokens Type: integerDefault: 400Range: 0 - 16000
Max new tokens
Output Schema

Output

Type: string

Example Execution Logs 
Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
Version Details
Version ID
0cc8616147b2fe24148d899688fb386ac9f3a2e19dba42c1d25732b7d4fb0f55
Version Created
October 26, 2023
Run on Replicate →