Using Jupyter Notebook¶

In the previous page, we showed an example of building a simple image understanding app. However, one-shotting the entire app in Python isn't very convenient. Instead, for non-trivial apps, you can interactively build the app logic in a Jupyter notebook. Here's an example that uses the built-in composite task MLLMTask.

In [1]:

from cornserve.task.builtins.mllm import MLLMTask, Modality
mllm = MLLMTask(
    model_id="Qwen/Qwen2-VL-7B-Instruct",
    modalities=[Modality.IMAGE],
)

from cornserve.task.builtins.mllm import MLLMTask, Modality mllm = MLLMTask( model_id="Qwen/Qwen2-VL-7B-Instruct", modalities=[Modality.IMAGE], )

You can deploy this task to the cluster so that you can later invoke it. To deploy, you need to create a CornserveClient.

In [2]:

from cornserve.frontend import CornserveClient
cornserve = CornserveClient(url="your-endpoint:port")

from cornserve.frontend import CornserveClient cornserve = CornserveClient(url="your-endpoint:port")

Connected to Cornserve gateway at your-endpoint:port

Deploy may take a few minutes at first.

In [3]:

cornserve.deploy(mllm)

cornserve.deploy(mllm)

Out[3]:

TaskResponse(status=200, content='Tasks declared used')

The builtin MLLMTask takes MLLMInput, so you can construct the input and try to invoke it. Note the multimodal_data field is a list of tuples.

In [4]:

from cornserve.task.builtins.mllm import MLLMInput
mllm_input = MLLMInput(
    prompt="What is this image about?",
    multimodal_data=[("image", "https://picsum.photos/seed/random/200/300")]
)

from cornserve.task.builtins.mllm import MLLMInput mllm_input = MLLMInput( prompt="What is this image about?", multimodal_data=[("image", "https://picsum.photos/seed/random/200/300")] )

In [5]:

await mllm(mllm_input)

await mllm(mllm_input)

INFO 2025-04-30 14:58:12,737 [cornserve.task.base:509] Dispatching tasks to http://your-endpoint:port/tasks/invoke

Out[5]:

MLLMOutput(response='The image shows a dog wrapped in a blanket. The blanket appears to be plaid or checkered, with a mix of colors including beige, pink, and green. The dog is sitting on the ground, and the blanket is wrapped around its body, covering its head and part of its back. The background suggests an outdoor setting, possibly a forest or a park, with trees and greenery visible. The overall scene gives a cozy and warm impression, as if the dog is being kept warm in the cold weather.')

Now we can build more complex Tasks. For example:

In [6]:

from cornserve.task.base import Task, TaskInput, TaskOutput
class MyTaskInput(TaskInput):
    """Task input model."""

    first_prompt: str
    second_prompt: str
    multimodal_data: list[tuple[str, str]] = []

class MyTaskOutput(TaskOutput):
    """Task output model."""
    response: str

class MyTask(Task):
    def __init__(self):
        super().__init__()
        self.mllm = MLLMTask(
            model_id="Qwen/Qwen2-VL-7B-Instruct",
            modalities=[Modality.IMAGE],
        )

    def invoke(self, input: MyTaskInput) -> MyTaskOutput:
        """Invoke the task with the given input."""
        mllm_input = MLLMInput(
            prompt=input.first_prompt,
            multimodal_data=input.multimodal_data,
        )
        mllm_output = self.mllm.invoke(mllm_input)


        input = MLLMInput(
            prompt=input.second_prompt,
            multimodal_data=input.multimodal_data,
        )

        output = self.mllm.invoke(input)
        return MyTaskOutput(response=f"{mllm_output.response} \n-----\n{output.response}")

from cornserve.task.base import Task, TaskInput, TaskOutput class MyTaskInput(TaskInput): """Task input model.""" first_prompt: str second_prompt: str multimodal_data: list[tuple[str, str]] = [] class MyTaskOutput(TaskOutput): """Task output model.""" response: str class MyTask(Task): def __init__(self): super().__init__() self.mllm = MLLMTask( model_id="Qwen/Qwen2-VL-7B-Instruct", modalities=[Modality.IMAGE], ) def invoke(self, input: MyTaskInput) -> MyTaskOutput: """Invoke the task with the given input.""" mllm_input = MLLMInput( prompt=input.first_prompt, multimodal_data=input.multimodal_data, ) mllm_output = self.mllm.invoke(mllm_input) input = MLLMInput( prompt=input.second_prompt, multimodal_data=input.multimodal_data, ) output = self.mllm.invoke(input) return MyTaskOutput(response=f"{mllm_output.response} \n-----\n{output.response}")

We can deploy it and test it. Note the deployment finishes instantly because Cornserve can reuse the UnitTasks in each Task.

In [7]:

my_task = MyTask()
cornserve.deploy(my_task)

my_task = MyTask() cornserve.deploy(my_task)

Out[7]:

TaskResponse(status=200, content='Tasks declared used')

In [8]:

my_input = MyTaskInput(
    first_prompt="What's the name of the flower?",
    second_prompt="Write a haiku about the image",
    multimodal_data=[("image", "https://upload.wikimedia.org/wikipedia/commons/thumb/f/fb/Crab_apple_flower_4004.jpg/2880px-Crab_apple_flower_4004.jpg")],
)
result = await my_task(my_input)
print(result.response)

my_input = MyTaskInput( first_prompt="What's the name of the flower?", second_prompt="Write a haiku about the image", multimodal_data=[("image", "https://upload.wikimedia.org/wikipedia/commons/thumb/f/fb/Crab_apple_flower_4004.jpg/2880px-Crab_apple_flower_4004.jpg")], ) result = await my_task(my_input) print(result.response)

INFO 2025-04-30 14:58:17,938 [cornserve.task.base:509] Dispatching tasks to http://your-endpoint:port/tasks/invoke

The flower in the picture is a Crab Apple (Malus). Crab apples are a type of apple tree known for their small, often colorful flowers that bloom in early spring. They are not typically grown for their fruit, which is often too small and sour for eating, but are valued for their ornamental value and the attractive flowers they produce. 
-----
Soft pink petals,
Gentle leaves whispering,
Spring's gentle breath.

We no longer need the mllm, so can teardown it in the cluster. Note this returns instantly because its UnitTask are also used by my_task!

In [9]:

cornserve.teardown(mllm)

cornserve.teardown(mllm)

Out[9]:

TaskResponse(status=200, content='Tasks declared not used')

The CornserveClient uses a web socket to talk to the Cornserve Gateway. A disconnect will automatically teardown all the tasks you deployed.

In [10]:

cornserve.close()

cornserve.close()

Closed connection to Cornserve gateway.
Closed keep-alive thread.