Skip to content

Tasks

Overview

A Task is a purpose-built abstraction for the Large Language Model (LLM). Griptape offers various types of Tasks, each suitable for specific use cases.

Context

Tasks that take input have a field input which lets you define the Task objective. Within the input, you can access the following context variables:

  • args: an array of arguments passed to the .run() method.
  • structure: the structure that the task belongs to.
  • user defined context variables

Additional context variables may be added based on the Structure running the task.

from griptape.structures import Agent
from griptape.tasks import PromptTask

agent = Agent()
agent.add_task(
    PromptTask(
        "Respond to the user's following question '{{ args[0] }}' in the language '{{preferred_language}}' and tone '{{tone}}'.",
        context={"preferred_language": "ENGLISH", "tone": "PLAYFUL"},
    )
)

agent.run("How do I bake a cake?")
[09/08/23 11:12:47] INFO     PromptTask 0f5a5def49864126834627b6140f3e63
                             Input: Respond to the user's following question 'How do I bake a cake?' in the language 'ENGLISH' and tone
                             'PLAYFUL'.
[09/08/23 11:13:17] INFO     PromptTask 0f5a5def49864126834627b6140f3e63
                             Output: Oh, you're in for a treat! Baking a cake is like creating a masterpiece, but way more delicious! Here's a
                             simple recipe to get you started:

                             1. Preheat your oven to 350°F (175°C). It's like sunbathing, but for your cake!

                             2. Grab a bowl and mix together 2 cups of sugar and 1/2 cup of softened butter. It's like making sweet, buttery
                             sandcastles!

                             3. Crack in 3 eggs, one at a time, and stir in 2 teaspoons of vanilla extract. It's a pool party in your bowl!

                             4. In a separate bowl, combine 1 1/2 cups of all-purpose flour, 1 3/4 teaspoons of baking powder, and a pinch of
                             salt. This is the dry gang!

                             5. Gradually mix the dry gang into the buttery pool party. Stir until it's just combined, we don't want to
                             overwork the partygoers!

                             6. Pour the batter into a greased cake pan. It's like tucking your cake into bed!

                             7. Bake for 30 to 40 minutes, or until a toothpick comes out clean. It's like playing hide and seek with your
                             cake!

                             8. Let it cool, then frost and decorate as you like. This is where you can let your creativity shine!

                             Remember, baking is all about having fun and enjoying the process. So, put on your favorite tunes, roll up your
                             sleeves, and let's get baking! 🍰🎉

Hooks

All Tasks implement RunnableMixin which provides on_before_run and on_after_run hooks for the Task lifecycle.

These hooks can be used to perform actions before and after the Task is run. For example, you can mask sensitive information before running the Task, and transform the output after the Task is run.

import json
import re

from griptape.structures import Agent
from griptape.tasks import PromptTask
from griptape.tasks.base_task import BaseTask

SSN_PATTERN = re.compile(r"\b\d{3}-\d{2}-\d{4}\b")

original_input = None


def on_before_run(task: BaseTask) -> None:
    global original_input

    original_input = task.input.value

    if isinstance(task, PromptTask):
        task.input = SSN_PATTERN.sub("xxx-xx-xxxx", task.input.value)


def on_after_run(task: BaseTask) -> None:
    if task.output is not None:
        task.output.value = json.dumps(
            {"original_input": original_input, "masked_input": task.input.value, "output": task.output.value}, indent=2
        )


agent = Agent(
    tasks=[
        PromptTask(
            "Respond to this user: {{ args[0] }}",
            on_before_run=on_before_run,
            on_after_run=on_after_run,
        )
    ]
)
agent.run("Hello! My favorite color is blue, and my social security number is 123-45-6789.")
[10/15/24 15:14:10] INFO     PromptTask 63a0c734059c42808c87dff351adc8ab
                             Input: Respond to this user: Hello! My favorite color is blue, and my social security number is xxx-xx-xxxx.
[10/15/24 15:14:11] INFO     PromptTask 63a0c734059c42808c87dff351adc8ab
                             Output: {
                               "original_input": "Respond to this user: Hello! My favorite color is blue, and my social security number is 123-45-6789.",
                               "masked_input": "Respond to this user: Hello! My favorite color is blue, and my social security number is xxx-xx-xxxx.",
                               "output": "Hello! It's great to hear that your favorite color is blue. However, it's important to keep your personal information, like your
                             social security number, private and secure. If you have any questions or need assistance, feel free to ask!"
                             }

Prompt Task

For general purpose prompting, use the PromptTask:

from griptape.structures import Agent
from griptape.tasks import PromptTask

agent = Agent()
agent.add_task(
    # take the first argument from the agent `run` method
    PromptTask("Respond to the following request: {{ args[0] }}"),
)

agent.run("Write me a haiku")
[10/20/23 15:27:26] INFO     PromptTask f5025c6352914e9f80ef730e5269985a        
                             Input: Respond to the following request:     
                             Write me a haiku                                   
[10/20/23 15:27:28] INFO     PromptTask f5025c6352914e9f80ef730e5269985a        
                             Output: Gentle morning dew,                        
                             Kisses the waking flowers,                         
                             Day begins anew.

If the model supports it, you can also pass image inputs:

from griptape.loaders import ImageLoader
from griptape.structures import Agent

agent = Agent()

image_artifact = ImageLoader().load("tests/resources/mountain.jpg")

agent.run([image_artifact, "What's in this image?"])
[06/21/24 10:01:08] INFO     PromptTask c229d1792da34ab1a7c45768270aada9
                             Input: What's in this image?

                             Media, type: image/jpeg, size: 82351 bytes
[06/21/24 10:01:12] INFO     PromptTask c229d1792da34ab1a7c45768270aada9
                             Output: The image depicts a stunning mountain landscape at sunrise or sunset. The sun is partially visible on the left side of the image,
                             casting a warm golden light over the scene. The mountains are covered with snow at their peaks, and a layer of clouds or fog is settled in the
                             valleys between them. The sky is a mix of warm colors near the horizon, transitioning to cooler blues higher up, with some scattered clouds
                             adding texture to the sky. The overall scene is serene and majestic, highlighting the natural beauty of the mountainous terrain.

Toolkit Task

To use Griptape Tools, use a Toolkit Task. This Task takes in one or more Tools which the LLM will decide to use through Chain of Thought (CoT) reasoning. Because this Task uses CoT, it is recommended to only use with very capable models.

from griptape.structures import Agent
from griptape.tasks import ToolkitTask
from griptape.tools import FileManagerTool, PromptSummaryTool, WebScraperTool

agent = Agent()
agent.add_task(
    ToolkitTask(
        "Load https://www.griptape.ai, summarize it, and store it in a file called griptape.txt",
        tools=[WebScraperTool(off_prompt=True), FileManagerTool(off_prompt=True), PromptSummaryTool(off_prompt=True)],
    ),
)

agent.run()
[08/12/24 15:16:30] INFO     ToolkitTask f5b44fe1dadc4e6688053df71d97e0de
                             Input: Load https://www.griptape.ai, summarize it, and store it in a file called griptape.txt
[08/12/24 15:16:32] INFO     Subtask a4483eddfbe84129b0f4c04ef0f5d695
                             Actions: [
                               {
                                 "tag": "call_AFeOL9MGhZ4mPFCULcBEm4NQ",
                                 "name": "WebScraperTool",
                                 "path": "get_content",
                                 "input": {
                                   "values": {
                                     "url": "https://www.griptape.ai"
                                   }
                                 }
                               }
                             ]
                    INFO     Subtask a4483eddfbe84129b0f4c04ef0f5d695
                             Response: Output of "WebScraperTool.get_content" was stored in memory with memory_name "TaskMemory" and artifact_namespace
                             "c6a6bcfc16f34481a068108aeaa6838e"
[08/12/24 15:16:33] INFO     Subtask ee5f11666ded4dc39b94e4c59d18fbc7
                             Actions: [
                               {
                                 "tag": "call_aT7DX0YSQPmOcnumWXrGoMNt",
                                 "name": "PromptSummaryTool",
                                 "path": "summarize",
                                 "input": {
                                   "values": {
                                     "summary": {
                                       "memory_name": "TaskMemory",
                                       "artifact_namespace": "c6a6bcfc16f34481a068108aeaa6838e"
                                     }
                                   }
                                 }
                               }
                             ]
[08/12/24 15:16:37] INFO     Subtask ee5f11666ded4dc39b94e4c59d18fbc7
                             Response: Output of "PromptSummaryTool.summarize" was stored in memory with memory_name "TaskMemory" and artifact_namespace
                             "669d29a704444176be93d09d014298df"
[08/12/24 15:16:38] INFO     Subtask d9b2dd9f96d841f49f5d460e33905183
                             Actions: [
                               {
                                 "tag": "call_QgMk1M1UuD6DAnxjfQz1MH6X",
                                 "name": "FileManagerTool",
                                 "path": "save_memory_artifacts_to_disk",
                                 "input": {
                                   "values": {
                                     "dir_name": ".",
                                     "file_name": "griptape.txt",
                                     "memory_name": "TaskMemory",
                                     "artifact_namespace": "669d29a704444176be93d09d014298df"
                                   }
                                 }
                               }
                             ]
                    INFO     Subtask d9b2dd9f96d841f49f5d460e33905183
                             Response: Successfully saved memory artifacts to disk
[08/12/24 15:16:39] INFO     ToolkitTask f5b44fe1dadc4e6688053df71d97e0de
                             Output: The content from https://www.griptape.ai has been summarized and stored in a file called `griptape.txt`.

Tool Task

Another way to use Griptape Tools, is with a Tool Task. This Task takes in a single Tool which the LLM will use without Chain of Thought (CoT) reasoning. Because this Task does not use CoT, it is better suited for less capable models.

from griptape.structures import Agent
from griptape.tasks import ToolTask
from griptape.tools import CalculatorTool

# Initialize the agent and add a task
agent = Agent()
agent.add_task(ToolTask(tool=CalculatorTool()))

# Run the agent with a prompt
agent.run("Give me the answer for 5*4.")
[10/20/23 14:20:25] INFO     ToolTask df1604b417a84ee781dbd1f2b904ed30          
                             Input: Give me the answer for 5*4.                 
[10/20/23 14:20:29] INFO     Subtask a9a9ad7be2bf465fa82bd350116fabe4           
                             Action: {                                          

                               "name": "CalculatorTool",                            
                               "path": "calculate",                         
                               "input": {                                       
                                 "values": {                                    
                                   "expression": "5*4"                          
                                 }                                              
                               }                                                
                             }                                                  
[10/20/23 14:20:30] INFO     Subtask a9a9ad7be2bf465fa82bd350116fabe4           
                             Response: 20                                    
                    INFO     ToolTask df1604b417a84ee781dbd1f2b904ed30          
                             Output: 20       

Extraction Task

To extract information from text, use an ExtractionTask. This Task takes an Extraction Engine, and a set of arguments specific to the Engine.

CSV Extraction

from griptape.drivers import OpenAiChatPromptDriver
from griptape.engines import CsvExtractionEngine
from griptape.structures import Agent
from griptape.tasks import ExtractionTask

# Instantiate the CSV extraction engine
csv_extraction_engine = CsvExtractionEngine(
    prompt_driver=OpenAiChatPromptDriver(model="gpt-3.5-turbo"), column_names=["Name", "Age", "Address"]
)

# Define some unstructured data and columns
csv_data = """
Alice, 28, lives in New York.
Bob, 35 lives in California.
Charlie is 40 and lives in Texas.
"""


# Create an agent and add the ExtractionTask to it
agent = Agent()
agent.add_task(
    ExtractionTask(
        extraction_engine=csv_extraction_engine,
    )
)

# Run the agent
agent.run(csv_data)
[12/19/23 10:33:11] INFO     ExtractionTask e87fb457edf8423ab8a78583badd7a11
                             Input:
                             Alice, 28, lives in New York.
                             Bob, 35 lives in California.
                             Charlie is 40 and lives in Texas.

[12/19/23 10:33:13] INFO     ExtractionTask e87fb457edf8423ab8a78583badd7a11
                             Output: Name,Age,Address
                             Alice,28,New York
                             Bob,35,California
                             Charlie,40,Texas

JSON Extraction

from schema import Schema

from griptape.drivers import OpenAiChatPromptDriver
from griptape.engines import JsonExtractionEngine
from griptape.structures import Agent
from griptape.tasks import ExtractionTask

# Instantiate the json extraction engine
json_extraction_engine = JsonExtractionEngine(
    prompt_driver=OpenAiChatPromptDriver(model="gpt-3.5-turbo"),
    template_schema=Schema({"users": [{"name": str, "age": int, "location": str}]}).json_schema("UserSchema"),
)

# Define some unstructured data and a schema
json_data = """
Alice (Age 28) lives in New York.
Bob (Age 35) lives in California.
"""

agent = Agent()
agent.add_task(
    ExtractionTask(
        extraction_engine=json_extraction_engine,
    )
)

# Run the agent
agent.run(json_data)
[12/19/23 10:37:41] INFO     ExtractionTask 3315cc77f94943a2a2dceccfe44f6a67
                             Input:
                             Alice (Age 28) lives in New York.
                             Bob (Age 35) lives in California.

[12/19/23 10:37:44] INFO     ExtractionTask 3315cc77f94943a2a2dceccfe44f6a67
                             Output: {'name': 'Alice', 'age': 28, 'location': 'New York'}
                             {'name': 'Bob', 'age': 35, 'location': 'California'}

Text Summary Task

To summarize a text, use the TextSummaryTask. This Task takes an Summarization Engine, and a set of arguments to the engine.

from griptape.structures import Agent
from griptape.tasks import TextSummaryTask

# Create a new agent
agent = Agent()

# Add the TextSummaryTask to the agent
agent.add_task(TextSummaryTask())


# Run the agent
agent.run(
    "Artificial Intelligence (AI) is a branch of computer science that deals with "
    "creating machines capable of thinking and learning. It encompasses various fields "
    "such as machine learning, neural networks, and deep learning. AI has the potential "
    "to revolutionize many sectors, including healthcare, finance, and transportation. "
    "Our life in this modern age depends largely on computers. It is almost impossible "
    "to think about life without computers. We need computers in everything that we use "
    "in our daily lives. So it becomes very important to make computers intelligent so "
    "that our lives become easy. Artificial Intelligence is the theory and development "
    "of computers, which imitates the human intelligence and senses, such as visual "
    "perception, speech recognition, decision-making, and translation between languages."
    " Artificial Intelligence has brought a revolution in the world of technology. "
)
[10/20/23 15:37:46] INFO     TextSummaryTask e870f2a6226f43fcb89f93b1c0c85b10   
                             Input: Artificial Intelligence (AI) is a branch of 
                             computer science that deals with creating machines 
                             capable of thinking and learning. It encompasses   
                             various fields such as machine learning, neural    
                             networks, and deep learning. AI has the potential  
                             to revolutionize many sectors, including           
                             healthcare, finance, and transportation. Our life  
                             in this modern age depends largely on computers. It
                             is almost impossible to think about life without   
                             computers. We need computers in everything that we 
                             use in our daily lives. So it becomes very         
                             important to make computers intelligent so that our
                             lives become easy. Artificial Intelligence is the  
                             theory and development of computers, which imitates
                             the human intelligence and senses, such as visual  
                             perception, speech recognition, decision-making,   
                             and translation between languages. Artificial      
                             Intelligence has brought a revolution in the world 
                             of technology.                                     
[10/20/23 15:37:49] INFO     TextSummaryTask e870f2a6226f43fcb89f93b1c0c85b10   
                             Output: Artificial Intelligence (AI) is a branch of
                             computer science that focuses on creating          
                             intelligent machines. It encompasses various fields
                             such as machine learning and neural networks. AI   
                             has the potential to revolutionize sectors like    
                             healthcare, finance, and transportation. It is     
                             essential to make computers intelligent to simplify
                             our daily lives. AI imitates human intelligence and
                             senses, bringing a revolution in technology.   

RAG Task

To query text, use the RagTask. This task takes a RAG Engine, and a set of arguments specific to the engine.

from griptape.artifacts import TextArtifact
from griptape.drivers import LocalVectorStoreDriver, OpenAiChatPromptDriver, OpenAiEmbeddingDriver
from griptape.engines.rag import RagEngine
from griptape.engines.rag.modules import PromptResponseRagModule, VectorStoreRetrievalRagModule
from griptape.engines.rag.stages import ResponseRagStage, RetrievalRagStage
from griptape.structures import Agent
from griptape.tasks import RagTask

# Initialize Embedding Driver and Vector Store Driver
vector_store_driver = LocalVectorStoreDriver(embedding_driver=OpenAiEmbeddingDriver())

artifacts = [
    TextArtifact("Griptape builds AI-powered applications that connect securely to your enterprise data and APIs."),
    TextArtifact("Griptape Agents provide incredible power and flexibility when working with large language models."),
]
vector_store_driver.upsert_text_artifacts({"griptape": artifacts})

# Instantiate the agent and add RagTask with the RagEngine
agent = Agent()
agent.add_task(
    RagTask(
        "Respond to the following query: {{ args[0] }}",
        rag_engine=RagEngine(
            retrieval_stage=RetrievalRagStage(
                retrieval_modules=[
                    VectorStoreRetrievalRagModule(
                        vector_store_driver=vector_store_driver, query_params={"namespace": "griptape", "top_n": 20}
                    )
                ]
            ),
            response_stage=ResponseRagStage(
                response_modules=[PromptResponseRagModule(prompt_driver=OpenAiChatPromptDriver(model="gpt-4o"))]
            ),
        ),
    )
)

# Run the agent with a query string
agent.run("Give me information about Griptape")

Code Execution Task

To execute an arbitrary Python function, use the CodeExecutionTask. This task takes a python function, and authors can elect to return a custom artifact.

from griptape.artifacts import BaseArtifact, TextArtifact
from griptape.structures import Pipeline
from griptape.tasks import CodeExecutionTask, PromptTask


def character_counter(task: CodeExecutionTask) -> BaseArtifact:
    result = len(task.input)
    # For functions that don't need to return anything, we recommend returning task.input
    return TextArtifact(str(result))


# Instantiate the pipeline
pipeline = Pipeline()

pipeline.add_tasks(
    # take the first argument from the pipeline `run` method
    CodeExecutionTask(on_run=character_counter),
    # # take the output from the previous task and insert it into the prompt
    PromptTask("{{args[0]}} using {{ parent_output }} characters"),
)

pipeline.run("Write me a line in a poem")
[01/09/24 15:23:54] INFO     CodeExecutionTask 048b1f548683475187064dde90055f72 
                             Input: Write me a line in a poem                   
                    INFO     CodeExecutionTask 048b1f548683475187064dde90055f72 
                             Output: 25                                         
                    INFO     PromptTask b6156dc5c0c6404488ab925989e78b01        
                             Input: Write me a line in a poem using 25          
                             characters                                         
[01/09/24 15:24:03] INFO     PromptTask b6156dc5c0c6404488ab925989e78b01        
                             Output: "Silent code, loud impact."  

Image Generation Tasks

To generate an image, use one of the following Image Generation Tasks. All Image Generation Tasks accept an Image Generation Engine configured to use an Image Generation Driver.

All successful Image Generation Tasks will always output an Image Artifact. Each task can be configured to additionally write the generated image to disk by providing either the output_file or output_dir field. The output_file field supports file names in the current directory (my_image.png), relative directory prefixes (images/my_image.png), or absolute paths (/usr/var/my_image.png). By setting output_dir, the task will generate a file name and place the image in the requested directory.

Prompt Image Generation Task

The Prompt Image Generation Task generates an image from a text prompt.

from griptape.drivers import OpenAiImageGenerationDriver
from griptape.engines import PromptImageGenerationEngine
from griptape.structures import Pipeline
from griptape.tasks import PromptImageGenerationTask

# Create a driver configured to use OpenAI's DALL-E 3 model.
driver = OpenAiImageGenerationDriver(
    model="dall-e-3",
    quality="hd",
    style="natural",
)

# Create an engine configured to use the driver.
engine = PromptImageGenerationEngine(
    image_generation_driver=driver,
)

# Instantiate a pipeline.
pipeline = Pipeline()

# Add a PromptImageGenerationTask to the pipeline.
pipeline.add_tasks(
    PromptImageGenerationTask(
        input="{{ args[0] }}",
        image_generation_engine=engine,
        output_dir="images/",
    )
)

pipeline.run("An image of a mountain on a summer day")

Variation Image Generation Task

The Variation Image Generation Task generates an image using an input image and a text prompt. The input image is used as a basis for generating a new image as requested by the text prompt.

from griptape.drivers import AmazonBedrockImageGenerationDriver, BedrockStableDiffusionImageGenerationModelDriver
from griptape.engines import VariationImageGenerationEngine
from griptape.loaders import ImageLoader
from griptape.structures import Pipeline
from griptape.tasks import VariationImageGenerationTask

# Create a driver configured to use Stable Diffusion via Bedrock.
driver = AmazonBedrockImageGenerationDriver(
    image_generation_model_driver=BedrockStableDiffusionImageGenerationModelDriver(),
    model="stability.stable-diffusion-xl-v0",
)

# Create an engine configured to use the driver.
engine = VariationImageGenerationEngine(
    image_generation_driver=driver,
)

# Load input image artifact.
image_artifact = ImageLoader().load("tests/resources/mountain.png")

# Instantiate a pipeline.
pipeline = Pipeline()

# Add a VariationImageGenerationTask to the pipeline.
pipeline.add_task(
    VariationImageGenerationTask(
        input=("{{ args[0] }}", image_artifact),
        image_generation_engine=engine,
        output_dir="images/",
    )
)

pipeline.run("An image of a mountain landscape on a snowy winter day")

Inpainting Image Generation Task

The Inpainting Image Generation Task generates an image using an input image, a mask image, and a text prompt. The input image will be modified within the bounds of the mask image as requested by the text prompt.

from griptape.drivers import AmazonBedrockImageGenerationDriver, BedrockStableDiffusionImageGenerationModelDriver
from griptape.engines import InpaintingImageGenerationEngine
from griptape.loaders import ImageLoader
from griptape.structures import Pipeline
from griptape.tasks import InpaintingImageGenerationTask

# Create a driver configured to use Stable Diffusion via Bedrock.
driver = AmazonBedrockImageGenerationDriver(
    image_generation_model_driver=BedrockStableDiffusionImageGenerationModelDriver(),
    model="stability.stable-diffusion-xl-v0",
)

# Create an engine configured to use the driver.
engine = InpaintingImageGenerationEngine(
    image_generation_driver=driver,
)

# Load input image artifacts.
image_artifact = ImageLoader().load("tests/resources/mountain.png")

mask_artifact = ImageLoader().load("tests/resources/mountain-mask.png")

# Instantiate a pipeline.
pipeline = Pipeline()

# Add an InpaintingImageGenerationTask to the pipeline.
pipeline.add_task(
    InpaintingImageGenerationTask(
        input=("{{ args[0] }}", image_artifact, mask_artifact), image_generation_engine=engine, output_dir="images/"
    )
)

pipeline.run("An image of a castle built into the side of a mountain")

Outpainting Image Generation Task

The Outpainting Image Generation Task generates an image using an input image, a mask image, and a text prompt. The input image will be modified outside the bounds of a mask image as requested by the text prompt.

from griptape.drivers import AmazonBedrockImageGenerationDriver, BedrockStableDiffusionImageGenerationModelDriver
from griptape.engines import OutpaintingImageGenerationEngine
from griptape.loaders import ImageLoader
from griptape.structures import Pipeline
from griptape.tasks import OutpaintingImageGenerationTask

# Create a driver configured to use Stable Diffusion via Bedrock.
driver = AmazonBedrockImageGenerationDriver(
    image_generation_model_driver=BedrockStableDiffusionImageGenerationModelDriver(),
    model="stability.stable-diffusion-xl-v0",
)

# Create an engine configured to use the driver.
engine = OutpaintingImageGenerationEngine(
    image_generation_driver=driver,
)

# Load input image artifacts.
image_artifact = ImageLoader().load("tests/resources/mountain.png")

mask_artifact = ImageLoader().load("tests/resources/mountain-mask.png")

# Instantiate a pipeline.
pipeline = Pipeline()

# Add an OutpaintingImageGenerationTask to the pipeline.
pipeline.add_task(
    OutpaintingImageGenerationTask(
        input=("{{ args[0] }}", image_artifact, mask_artifact),
        image_generation_engine=engine,
        output_dir="images/",
    )
)

pipeline.run("An image of a mountain shrouded by clouds")

Image Query Task

The Image Query Task performs a natural language query on one or more input images. This Task uses an Image Query Engine configured with an Image Query Driver to perform the query. The functionality provided by this Task depend on the capabilities of the model provided by the Driver.

This Task accepts two inputs: a query (represented by either a string or a Text Artifact) and a list of Image Artifacts or a Callable returning these two values.

from griptape.drivers import OpenAiImageQueryDriver
from griptape.engines import ImageQueryEngine
from griptape.loaders import ImageLoader
from griptape.structures import Pipeline
from griptape.tasks import ImageQueryTask

# Create a driver configured to use OpenAI's GPT-4 Vision model.
driver = OpenAiImageQueryDriver(
    model="gpt-4o",
    max_tokens=100,
)

# Create an engine configured to use the driver.
engine = ImageQueryEngine(
    image_query_driver=driver,
)

# Load the input image artifact.
image_artifact = ImageLoader().load("tests/resources/mountain.png")

# Instantiate a pipeline.
pipeline = Pipeline()

# Add an ImageQueryTask to the pipeline.
pipeline.add_task(
    ImageQueryTask(
        input=("{{ args[0] }}", [image_artifact]),
        image_query_engine=engine,
    )
)

pipeline.run("Describe the weather in the image")

Structure Run Task

The Structure Run Task runs another Structure with a given input. This Task is useful for orchestrating multiple specialized Structures in a single run. Note that the input to the Task is a tuple of arguments that will be passed to the Structure.

import os

from griptape.drivers import GoogleWebSearchDriver, LocalStructureRunDriver
from griptape.rules import Rule, Ruleset
from griptape.structures import Agent, Pipeline
from griptape.tasks import StructureRunTask
from griptape.tools import (
    PromptSummaryTool,
    WebScraperTool,
    WebSearchTool,
)


def build_researcher() -> Agent:
    return Agent(
        tools=[
            WebSearchTool(
                web_search_driver=GoogleWebSearchDriver(
                    api_key=os.environ["GOOGLE_API_KEY"],
                    search_id=os.environ["GOOGLE_API_SEARCH_ID"],
                ),
            ),
            WebScraperTool(
                off_prompt=True,
            ),
            PromptSummaryTool(off_prompt=False),
        ],
        rulesets=[
            Ruleset(
                name="Position",
                rules=[
                    Rule(
                        value="Senior Research Analyst",
                    )
                ],
            ),
            Ruleset(
                name="Objective",
                rules=[
                    Rule(
                        value="Uncover cutting-edge developments in AI and data science",
                    )
                ],
            ),
            Ruleset(
                name="Background",
                rules=[
                    Rule(
                        value="""You work at a leading tech think tank.,
                        Your expertise lies in identifying emerging trends.
                        You have a knack for dissecting complex data and presenting actionable insights."""
                    )
                ],
            ),
            Ruleset(
                name="Desired Outcome",
                rules=[
                    Rule(
                        value="Full analysis report in bullet points",
                    )
                ],
            ),
        ],
    )


def build_writer() -> Agent:
    return Agent(
        input="Instructions: {{args[0]}}\nContext: {{args[1]}}",
        rulesets=[
            Ruleset(
                name="Position",
                rules=[
                    Rule(
                        value="Tech Content Strategist",
                    )
                ],
            ),
            Ruleset(
                name="Objective",
                rules=[
                    Rule(
                        value="Craft compelling content on tech advancements",
                    )
                ],
            ),
            Ruleset(
                name="Backstory",
                rules=[
                    Rule(
                        value="""You are a renowned Content Strategist, known for your insightful and engaging articles.
                        You transform complex concepts into compelling narratives."""
                    )
                ],
            ),
            Ruleset(
                name="Desired Outcome",
                rules=[
                    Rule(
                        value="Full blog post of at least 4 paragraphs",
                    )
                ],
            ),
        ],
    )


team = Pipeline(
    tasks=[
        StructureRunTask(
            (
                """Perform a detailed examination of the newest developments in AI as of 2024.
                Pinpoint major trends, breakthroughs, and their implications for various industries.""",
            ),
            driver=LocalStructureRunDriver(create_structure=build_researcher),
        ),
        StructureRunTask(
            (
                """Utilize the gathered insights to craft a captivating blog
                article showcasing the key AI innovations.
                Ensure the content is engaging yet straightforward, appealing to a tech-aware readership.
                Keep the tone appealing and use simple language to make it less technical.""",
                "{{parent_output}}",
            ),
            driver=LocalStructureRunDriver(create_structure=build_writer),
        ),
    ],
)

team.run()

Text to Speech Task

This Task enables Structures to synthesize speech from text using Text to Speech Engines and Text to Speech Drivers.

import os

from griptape.drivers import ElevenLabsTextToSpeechDriver
from griptape.engines import TextToSpeechEngine
from griptape.structures import Pipeline
from griptape.tasks import TextToSpeechTask

driver = ElevenLabsTextToSpeechDriver(
    api_key=os.environ["ELEVEN_LABS_API_KEY"],
    model="eleven_multilingual_v2",
    voice="Matilda",
)

task = TextToSpeechTask(
    text_to_speech_engine=TextToSpeechEngine(
        text_to_speech_driver=driver,
    ),
)

Pipeline(tasks=[task]).run("Generate audio from this text: 'Hello, world!'")

Audio Transcription Task

This Task enables Structures to transcribe speech from text using Audio Transcription Engines and Audio Transcription Drivers.

from griptape.drivers import OpenAiAudioTranscriptionDriver
from griptape.engines import AudioTranscriptionEngine
from griptape.loaders import AudioLoader
from griptape.structures import Pipeline
from griptape.tasks import AudioTranscriptionTask

driver = OpenAiAudioTranscriptionDriver(model="whisper-1")

task = AudioTranscriptionTask(
    input=lambda _: AudioLoader().load("tests/resources/sentences2.wav"),
    audio_transcription_engine=AudioTranscriptionEngine(
        audio_transcription_driver=driver,
    ),
)

Pipeline(tasks=[task]).run()