如何在c#中读取一个非常大的JSON文件到一个数组中,以便拆分以供以后处理?


我已经设法得到一些工作,将:

读取文件忽略头文件,只将值读入数组。 在数组的每一行上放置一定数量的值。(所以我 可以稍后将其分割成一个放入2d数组)

这是用下面的代码完成的,但是在数组中输入几行后程序就崩溃了。这可能与文件大小有关。

// If the file extension was a jave file the following 
// load method will be use else it will move on to the 
// next else if statement
if (fileExtension == ".json") 
{
    int count = 0;
    int count2 = 0;
    int inOrOut = 0;
    int nRecords=1; 
    JsonTextReader reader = new JsonTextReader(new StreamReader(txtLoaction.Text));
    string[] rawData = new string[5];
    while (reader.Read())
    {
        if (reader.Value != null)
            if (inOrOut == 1)
            {
                if (count == 6)
                {
                    nRecords++;
                    Array.Resize(ref rawData, nRecords);
                    //textBox1.Text += "\r\n";
                    count = 0;
                }
                rawData[count2] += reader.Value + ","; //+"\r\n"
                inOrOut = 0;
                count++;
                if (count2 == 500)
                {
                    MessageBox.Show(rawData[499]);
                }
            }
            else
            {
                inOrOut = 1;
            }
    } 
}

我正在使用的JSON片段是:

[ 
    { "millis": "1000", 
      "stamp": "1273010254", 
      "datetime": "2010/5/4 21:57:34", 
      "light": "333", 
      "temp": "78.32", 
      "vcc": "3.54" }, 
] 

我需要这个JSON的值。例如,我需要“3.54”,但我不希望它打印“vcc”。

如何读取JSON文件,只提取需要放入数组的数据?


当前回答

使用开源库Cinchoo ETL,解析非常大的JSON文件是迭代的,使用简单

1. 动态方法:—不需要POCO类

        string json = @"
[
  {
    ""millis"": ""1000"",
    ""stamp"": ""1273010254"",
    ""datetime"": ""2010/5/4 21:57:34"",
    ""light"": ""333"",
    ""temp"": ""78.32"",
    ""vcc"": ""3.54""
  },
  {
    ""millis"": ""2000"",
    ""stamp"": ""1273010254"",
    ""datetime"": ""2010/5/4 21:57:34"",
    ""light"": ""333"",
    ""temp"": ""78.32"",
    ""vcc"": ""3.54""
  }
] 
";
        
        using (var r = ChoJSONReader.LoadText(json))
        {
            foreach (var rec in r)
                Console.WriteLine(rec.Dump());
        }

样本提琴:https://dotnetfiddle.net/mo1qvw

2. POCO:受压迫的

定义匹配json属性的POCO类

public class Item
{
    public int Millis { get; set; }
    public string Stamp { get; set; }
    public DateTime Datetime { get; set; }
    public string Light { get; set; }
    public float Temp { get; set; }
    public float Vcc { get; set; }
}

然后使用解析器加载JSON,如下所示

        string json = @"
[
  {
    ""millis"": ""1000"",
    ""stamp"": ""1273010254"",
    ""datetime"": ""2010/5/4 21:57:34"",
    ""light"": ""333"",
    ""temp"": ""78.32"",
    ""vcc"": ""3.54""
  },
  {
    ""millis"": ""2000"",
    ""stamp"": ""1273010254"",
    ""datetime"": ""2010/5/4 21:57:34"",
    ""light"": ""333"",
    ""temp"": ""78.32"",
    ""vcc"": ""3.54""
  }
] 
";
        
        using (var r = ChoJSONReader<Item>.LoadText(json))
        {
            foreach (var rec in r)
                Console.WriteLine(ChoUtility.Dump(rec));
        }

样本提琴:https://dotnetfiddle.net/fRWu0w

免责声明:我是这个库的作者。

其他回答

有一种更快的解析json的方法。网如果你使用的是。net core 3.0或更高版本,那么你可以使用System.Text.Json nuget包来序列化或反序列化。

您需要补充:

using System.Text.Json

然后你可以序列化为:

var jsonStr = JsonSerializer.Serialize(model);

反序列化为:

var model = JsonSerializer.Deserialize(jsonStr);
string jsonFilePath = @"C:\MyFolder\myFile.json";
            
string json = File.ReadAllText(jsonFilePath);
Dictionary<string, object> json_Dictionary = (new JavaScriptSerializer()).Deserialize<Dictionary<string, object>>(json);

foreach (var item in json_Dictionary)
{
    // parse here
}

这也可以通过以下方式来实现:

JObject data = JObject.Parse(File.ReadAllText(MyFilePath));

有一个更简单的方法从文件或从Web获取JSON: Json.Net.Curl

安装包Json.Net.Curl

// get JObject from local file system 
var json = Json.Net.Curl.Get(@"data\JObjectUnitTest1.json");
var json = await Json.Net.Curl.GetAsync(@"data\JObjectUnitTest1.json")


// get JObject from Server  
var json = await Json.Net.Curl.GetAsync("http://myserver.com/data.json");

GitHub项目 Nuget

Doing this yourself is an awful idea. Use Json.NET. It has already solved the problem better than most programmers could if they were given months on end to work on it. As for your specific needs, parsing into arrays and such, check the documentation, particularly on JsonTextReader. Basically, Json.NET handles JSON arrays natively and will parse them into strings, ints, or whatever the type happens to be without prompting from you. Here is a direct link to the basic code usages for both the reader and the writer, so you can have that open in a spare window while you're learning to work with this.

这是最好的:这次偷懒,使用一个库,这样你就永远解决了这个常见的问题。