Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rest api cannot return complete data #2923

Open
aaa1115910 opened this issue Mar 6, 2023 · 6 comments · May be fixed by #2925
Open

Rest api cannot return complete data #2923

aaa1115910 opened this issue Mar 6, 2023 · 6 comments · May be fixed by #2925

Comments

@aaa1115910
Copy link

  • TShock version: 5.1.3.0

Reproduction steps (if applicable)?

When returning data through RestCommand, the connection may be interrupted while transferring data, especially when the size of the transferred data is slightly larger.

This problem cannot be reproduced when testing locally, it only occurs when testing between multiple devices, such as the server and the player's computer.

Any stack traces or error messages (if known)?

An exception occurs when I use java to request api java.net.SocketException: Connection reset
And prompt net::ERR_CONTENT_LENGTH_MISMATCH in Chrome. The returned data is incomplete, only the first half of the data

I have tried to use the HttpListener in HttpServer.dll to create a server for testing, and found that the problem occurs when the returned data is too long, but the server created with System.Net.HttpListener returns the same data normally.

@sgkoishi
Copy link
Contributor

sgkoishi commented Mar 6, 2023

In China, all Internet Service Providers must enforce compliance checks, and licenses are required for all HTTP services. Your connection will be aborted during transmission if they are detected before completion. It is said that such checks were deployed at the IDC level, so probably no software-level solution.

@aaa1115910
Copy link
Author

In China, all Internet Service Providers must enforce compliance checks, and licenses are required for all HTTP services. Your connection will be aborted during transmission if they are detected before completion. It is said that such checks were deployed at the IDC level, so probably no software-level solution.

But even the two devices under the local router still have this problem

@sgkoishi
Copy link
Contributor

sgkoishi commented Mar 6, 2023

Unable to reproduce. Could you share the code of your testing server?
The snippet I tested can handle hundreds of megs without any problem.

using HttpServer;
using HttpServer.Headers;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System.Net;
using System.Text;

var listener = HttpServer.HttpListener.Create(IPAddress.Parse("0.0.0.0"), 8023);

listener.RequestReceived += OnRequest;
listener.Start(int.MaxValue);
Console.WriteLine("Server started");
Console.ReadLine();

static void OnRequest(object? sender, RequestEventArgs e)
{
    var obj = GenerateResp();
    var str = JsonConvert.SerializeObject(obj, Newtonsoft.Json.Formatting.Indented);
    e.Response.Connection.Type = ConnectionType.Close;
    e.Response.ContentType = new ContentTypeHeader("application/json; charset=utf-8");
    e.Response.Add(new StringHeader("Server", "Test/1.0"));
    var bytes = Encoding.UTF8.GetBytes(str);
    e.Response.Body.Write(bytes, 0, bytes.Length);
    e.Response.Status = HttpStatusCode.OK;
}

static JObject GenerateResp()
{
    var result = new JObject();
    for (var i = 0; i < 100; i++)
    {
        var bytes = new byte[1048576];
        Random.Shared.NextBytes(bytes);
        result.Add("key" + i, Convert.ToBase64String(bytes));
    }
    return result;
}

Update:
image
Seems Chrome on Android (left) is throttled (Firefox Android, the right one, works perfectly), however, it is not quite related to connection reset.

@aaa1115910
Copy link
Author

Here is the plugin code where I'm having trouble

using Rests;
using Terraria;
using TerrariaApi.Server;
using TShockAPI;

namespace TShockRestProblem;

[ApiVersion(2, 1)]
public class TestPlugin : TerrariaPlugin
{
    public override string Name => "Test Plugin";

    public TestPlugin(Main game) : base(game)
    {
    }

    public override void Initialize()
    {
        TShock.RestApi.Register(new RestCommand("/test", TestFunc));
    }

    private object TestFunc(RestRequestArgs args)
    {
        var result = new List<ResultItem>();
        for (var i = 0; i < 40; i++)
        {
            result.Add(ResultItem.CreateRandom());
        }

        return new RestObject
        {
            { "result", result }
        };
    }
}

public class ResultItem
{
    public int Id;
    public string Name;
    public string Description;
    public List<ItemData> Price;
    public List<ItemData> Unlock;

    public static ResultItem CreateRandom()
    {
        return new ResultItem
        {
            Id = RandomNumber(1000),
            Name = RandomString(10),
            Description = RandomString(100),
            Price = RandomItemDataList(RandomNumber(5)),
            Unlock = RandomItemDataList(RandomNumber(5))
        };
    }

    private static int RandomNumber(int max)
    {
        return new Random().Next(max);
    }

    private static string RandomString(int length)
    {
        const string chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789   ";
        return new string(Enumerable.Repeat(chars, length)
            .Select(s => s[new Random().Next(s.Length)]).ToArray());
    }

    private static List<ItemData> RandomItemDataList(int size)
    {
        var list = new List<ItemData>();
        for (var i = 0; i < size; i++)
        {
            list.Add(new ItemData(RandomString(10), RandomNumber(1000), RandomNumber(1000)));
        }

        return list;
    }
}

public class ItemData
{
    public string Name = "";
    public int Id;
    public int Stack = 1;
    public string Cmd = "";

    public ItemData(string name = "", int id = 0, int stack = 1)
    {
        Name = name;
        Id = id;
        Stack = stack;
    }
}

image

@aaa1115910
Copy link
Author

Unable to reproduce. Could you share the code of your testing server? The snippet I tested can handle hundreds of megs without any problem.

using HttpServer;
using HttpServer.Headers;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System.Net;
using System.Text;

var listener = HttpServer.HttpListener.Create(IPAddress.Parse("0.0.0.0"), 8023);

listener.RequestReceived += OnRequest;
listener.Start(int.MaxValue);
Console.WriteLine("Server started");
Console.ReadLine();

static void OnRequest(object? sender, RequestEventArgs e)
{
    var obj = GenerateResp();
    var str = JsonConvert.SerializeObject(obj, Newtonsoft.Json.Formatting.Indented);
    e.Response.Connection.Type = ConnectionType.Close;
    e.Response.ContentType = new ContentTypeHeader("application/json; charset=utf-8");
    e.Response.Add(new StringHeader("Server", "Test/1.0"));
    var bytes = Encoding.UTF8.GetBytes(str);
    e.Response.Body.Write(bytes, 0, bytes.Length);
    e.Response.Status = HttpStatusCode.OK;
}

static JObject GenerateResp()
{
    var result = new JObject();
    for (var i = 0; i < 100; i++)
    {
        var bytes = new byte[1048576];
        Random.Shared.NextBytes(bytes);
        result.Add("key" + i, Convert.ToBase64String(bytes));
    }
    return result;
}

Update: image Seems Chrome on Android (left) is throttled (Firefox Android, the right one, works perfectly), however, it is not quite related to connection reset.

I have no problem running your code, and my code is placed in the previous comment.

sgkoishi added a commit to sgkoishi/TShock that referenced this issue Mar 8, 2023
@sgkoishi
Copy link
Contributor

sgkoishi commented Mar 8, 2023

Reproduced. It seems that the library is not handling Connection: Close properly.

e.Response.Connection.Type = ConnectionType.Close;

@sgkoishi sgkoishi linked a pull request Mar 8, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants