无法使用新的 Dotnet 库发送在 wwwroot 下的本地图像到 Openai 。
原标题:Unable to send a local image that isn t under wwwroot to openai using its new dotnet library
I have an ASP.NET Core 6 Web API with the new official library from OpenAI (https://github.com/openai/openai-dotnet).
What I m trying to do, is to use a local image file to OpenAI. The file isn t under wwwroot, but under backend/assets/1.jpg.
I ve written a basic service to setup all the information needed in order to send a request to OpenAI. But the problem is that I m not able to send the image.
I keep getting errors like "url is too long" or "invalid image",
Here is my code - OpenAiService:
using OpenAI.Chat;
namespace backend.Services
{
public class OpenAiService
{
private readonly ChatClient _chatClient;
private readonly ChatCompletionOptions _options;
public OpenAiService(IConfiguration configuration)
{
var apiKey = configuration.GetValue("OpenAI:Key");
_chatClient = new ChatClient("gpt-4o", apiKey);
_options = new ChatCompletionOptions()
{
MaxTokens = 300,
};
}
public async Task ExtractListOfItems()
{
var imagePath = Path.Combine(Directory.GetCurrentDirectory(), "Assets", "1.jpg");
var localUrl = $"https://localhost:7068/assets/{Path.GetFileName(imagePath)}";
var messages = new List
{
new UserChatMessage(new List
{
ChatMessageContentPart.CreateTextMessageContentPart("Extract the items from the following image and return a list of items including prices and amount."),
ChatMessageContentPart.CreateImageMessageContentPart(new Uri(localUrl))
})
};
var completion = await _chatClient.CompleteChatAsync(messages, _options);
return completion.Value.ToString();
}
}
}
Demo controller for testing:
using Microsoft.AspNetCore.Mvc;
using System.Threading.Tasks;
using backend.Services;
using OpenAI;
using OpenAI.Chat;
namespace backend.Controllers;
[ApiController]
[Route("[controller]")]
public class OpenAiDemoController : ControllerBase
{
private readonly OpenAiService _openAiService;
public OpenAiDemoController(OpenAiService openAiService)
{
_openAiService = openAiService;
}
[HttpPost]
[Route("extract-items")]
public async Task CompleteSentence()
{
var completion = await _openAiService.ExtractListOfItems();
return Ok(completion);
}
}
program.cs file:
using backend.Configurations;
using backend.Services;
using Microsoft.Extensions.FileProviders;
var builder = WebApplication.CreateBuilder(args);
// Add services to the container.
builder.Services.Configure(builder.Configuration.GetSection("OpenAI"));
//add services
builder.Services.AddSingleton();
builder.Services.AddControllers();
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();
// builder.Services.AddScoped();
builder.Services.AddCors(opt =>
{
opt.AddPolicy("AllowAll", builder =>
{
builder.AllowAnyOrigin()
.AllowAnyMethod()
.AllowAnyHeader();
});
});
var app = builder.Build();
app.UseStaticFiles(new StaticFileOptions
{
FileProvider = new PhysicalFileProvider(
Path.Combine(builder.Environment.ContentRootPath, "Assets")),
RequestPath = "/assets"
});
app.UseStaticFiles(); // This serves files from wwwroot
app.UseStaticFiles(new StaticFileOptions
{
FileProvider = new PhysicalFileProvider(
Path.Combine(builder.Environment.ContentRootPath, "Assets")),
RequestPath = "/assets"
});
// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
app.UseSwagger();
app.UseSwaggerUI();
}
app.UseHttpsRedirection();
app.UseCors("AllowAll");
app.UseAuthorization();
app.MapControllers();
app.Run();
Any idea what I m doing wrong?
最佳回答
I was able to solve it on my own. There s an example in openai official repo that helped me to solve it.
https://github.com/openai/openai-dotnet/blob/main/examples/Chat/Example05_ChatWithVisionAsync.cs
Tho, im not sure if my implementation is mostly correct. I will leave this open for any other suggestions.
service:
using OpenAI.Chat;
namespace backend.Services
{
public class OpenAiService
{
private readonly ChatClient _chatClient;
private readonly ChatCompletionOptions _options;
public OpenAiService(IConfiguration configuration)
{
var apiKey = configuration.GetValue("OpenAI:Key");
_chatClient = new ChatClient("gpt-4o", apiKey);
_options = new ChatCompletionOptions()
{
MaxTokens = 300,
};
}
public async Task ExtractListOfItems()
{
var imageFilePath = Path.Combine("Assets", "1.jpg");
await using Stream imageStream = File.OpenRead(imageFilePath);
var imageBytes = BinaryData.FromStream(imageStream);
var messages = new List
{
new UserChatMessage(new List
{
ChatMessageContentPart.CreateTextMessageContentPart("describe the image. "),
ChatMessageContentPart.CreateImageMessageContentPart(imageBytes, "image/png")
})
};
var completion = await _chatClient.CompleteChatAsync(messages, _options);
return completion.Value.ToString();
}
}
}
usage in controller:
using Microsoft.AspNetCore.Mvc;
using System.Threading.Tasks;
using backend.Services;
using OpenAI;
using OpenAI.Chat;
namespace backend.Controllers;
[ApiController]
[Route("[controller]")]
public class OpenAiDemoController : ControllerBase
{
private readonly OpenAiService _openAiService;
public OpenAiDemoController(OpenAiService openAiService)
{
_openAiService = openAiService;
}
[HttpPost]
[Route("extract-items")]
public async Task CompleteSentence()
{
var completion = await _openAiService.ExtractListOfItems();
return Ok(completion);
}
}
no need to apply for static file middleware.
问题回答
暂无回答
相关问题
What is the use of `default` keyword in C#?
What is the use of default keyword in C#?
Is it introduced in C# 3.0 ?
Anyone feel like passing it forward?
I m the only developer in my company, and am getting along well as an autodidact, but I know I m missing out on the education one gets from working with and having code reviewed by more senior devs.
...
NSArray s, Primitive types and Boxing Oh My!
I m pretty new to the Objective-C world and I have a long history with .net/C# so naturally I m inclined to use my C# wits.
Now here s the question: I feel really inclined to create some type of ...
C# Marshal / Pinvoke CBitmap?
I cannot figure out how to marshal a C++ CBitmap to a C# Bitmap or Image class.
My import looks like this:
[DllImport(@"test.dll", CharSet = CharSet.Unicode)]
public static extern IntPtr ...
ADO.NET Entity Framework Association of Entities by Value Range
I have two EF entities. One has a property called HouseNumber. The other has two properties, one called StartHouseNumber and one called EndHouseNumber.
I want to create a many to many association ...
How to Use Ghostscript DLL to convert PDF to PDF/A
How to user GhostScript DLL to convert PDF to PDF/A. I know I kind of have to call the exported function of gsdll32.dll whose name is gsapi_init_with_args, but how do i pass the right arguments? BTW, ...
What is the most efficient keyvalue pair for ordering?
Since I cannot order my dictionary, what is the best way of going about taking key value pairs and also maintaing an index?
Linqy no matchy
Maybe it s something I m doing wrong. I m just learning Linq because I m bored. And so far so good. I made a little program and it basically just outputs all matches (foreach) into a label control.
...