===============================================================================
=                                 JSONLab                                     =
=           An open-source MATLAB/Octave JSON encoder and decoder             =
===============================================================================

* Copyright (C) 2011-2019  Qianqian Fang <q.fang at neu.edu>
* License: BSD or GNU General Public License version 3 (GPL v3), see License*.txt
* Version: 1.9.8 (Magnus - beta)
* JData Specification Version: Draft 2 (http://github.com/fangq/jdata)
* URL: http://openjdata.org/jsonlab

-------------------------------------------------------------------------------

Table of Content:

0.  What's New
I.  Introduction
II. Installation
III.Using JSONLab
IV. Known Issues and TODOs
V.  Contribution and feedback
V.  Acknowledgement

-------------------------------------------------------------------------------

0. What's New

JSONLab v1.9.8 is the beta release of the next milestone - code named "Magnus".

Starting from this release, JSONLab supports encoding/decoding MessagePack,
a widely-used binary JSON-like data format. Via ZMat v0.9, JSONLab v1.9.8
also supports LZMA/LZ4/LZ4HC data compression/decompression. More importantly,
JSONLab is now the official reference implementation for JData Specification (Draft 2)
as defined in http://github.com/fangq/jdata, the foundation for the OpenJData
Project (http://openjdata.org).

There have been numerous major updates to this toolbox since the previous 
release v1.9 in May 2019. A list of the major changes are summarized below
with key features marked by *:

* 2019-10-22*[650b5ec] enable preencode by default for savejson and saveubjson
* 2019-10-21*[874945f] decode graph data, encode non-char-keyed map data
* 2019-10-18 [11712b7] add any2jd, pass opt to name check, add more options
* 2019-10-18*[f97de9b] extract name encoding/decoding to separate function, like in easyh5
* 2019-10-17*[9d0fd4a] rewrite jdataencode
* 2019-10-15 [23f14d6] minor updates to make msgpack to work on octave
* 2019-09-16*[689cb40] support lz4 and lz4hc compression via zmat v0.9
* 2019-07-11*[06d33aa] update zmat test to support zmat v0.8 mox-the-fox
* 2019-06-24*[eba4078] saving table objects with new syntax
* 2019-06-12 [3eb6d56] change ArrayCompression keywords to ArrayZip to be short
* 2019-06-12*[e5f2ffb] complete saveubjson debug mode, add compression example
* 2019-06-11 [ebbcfd2] pass formatversion tag to jdatadecode
* 2019-06-10*[95b2eb0] add jdataencode and jdatadecode
* 2019-06-10*[f86219d] major update: use row-major for N-D array, incompatible with old JSONLab
* 2019-05-31*[0c467ee] support lzma and lzip compression decompression via zmat toolbox (http://github.com/fangq/zmat)
* 2019-05-31 [599ee4c] support categorical data
* 2019-05-30*[d47be45] fast bracket matching
* 2019-05-24*[0ec2d01] rewriting fastarrayparser, 10x faster for Octave, close #4 with fast bracket matching
* 2019-05-22*[d8c19b8] add support to MessagePack, close #53, add NestArray option, close #6
* 2019-05-19*[c87e7d2] support containers.Map


Please note that JSONLab v1.9.8 is compliant with JData Spec Draft 2, while
v1.9 and previous releases are compatible with Draft 1. The main differences are

* _ArrayCompressionMethod_, _ArrayCompressionSize_</tt> and <tt>_ArrayCompressedData_</tt> \
 are replaced by <tt>_ArrayZipType_``, <tt>_ArrayZipSize_</tt> and <tt>_ArrayZipData_``, respectively
* The serialization of N-D array data stored in <tt>_ArrayData_``is changed from \
 column-major to row-major

To read data files generated by JSONLab v1.9 or older versions, you need to attach
option <tt>'FormatVersion', 1.9</tt> in all the loadjson/savejson function calls. 
To convert an older file (JSON/UBJSON) to the new format, you should run

   data=loadjson('my_old_data_file.json','FormatVersion',1.9)
   savejson('',data,'FileName','new_file.json')

You are strongly encouraged to convert all previously generated data files using the new
format.


-------------------------------------------------------------------------------

I.  Introduction

JSONLab is a free and open-source implementation of a JSON/UBJSON/MessagePack encoder 
and a decoder in the native MATLAB language. It can be used to convert a MATLAB 
data structure (array, struct, cell, struct array, cell array, and objects) into 
JSON/UBJSON/MessagePack formatted strings, or to decode a 
JSON/UBJSON/MessagePack file into MATLAB data structure. JSONLab supports both 
MATLAB and [http://www.gnu.org/software/octave GNU Octave] (a free MATLAB clone).

JSON ([http://www.json.org/ JavaScript Object Notation]) is a highly portable, 
human-readable and [http://en.wikipedia.org/wiki/JSON "fat-free"] text format 
to represent complex and hierarchical data. It is as powerful as [http://en.wikipedia.org/wiki/XML XML]
but less verbose. JSON format is widely used for data-exchange in applications.

UBJSON ([http://ubjson.org/ Universal Binary JSON]) is a binary JSON format, specifically 
specifically optimized for compact file size and better performance while keeping
the semantics as simple as the text-based JSON format. Using the UBJSON
format allows to wrap complex binary data in a flexible and extensible
structure, making it possible to process complex and large dataset 
without accuracy loss due to text conversions. MessagePack is another binary
JSON-like data format widely used in data exchange in web/native applications.
It is slightly more compact than UBJSON, but is not directly readable compared
to UBJSON.

We envision that both JSON and its binary counterparts will play important 
roles as mainstream data-exchange formats for scientific research.
It has both the flexibility and generality as offered by other popular 
general-purpose file specifications, such as [http://www.hdfgroup.org/HDF5/whatishdf5.html HDF5] 
but with significantly reduced complexity and excellent readability.

Towards this goal, we have developed the JData Specification (http://github.com/fangq/jdata) 
to standardize serializations of complex scientific data structures, such as
N-D arrays, sparse/complex-valued arrays, trees, maps, tables and graphs using
JSON/binary JSON constructs. The text and binary formatted JData files are
syntactically compatible with JSON/UBJSON formats, and can be readily parsed 
using existing JSON and UBJSON parsers.

Please note that data files produced by <tt>saveubjson</tt> may utilize a special
"optimized header" to store N-D (N>1) arrays, as defined in the JData Specification Draft 2.
This feature is not supported by UBJSON Specification Draft 12. To produce 
UBJSON files that can be parsed by UBJSON-Draft-12 compliant parsers, you must
add the option <tt>'NestArray',1</tt> in the call to <tt>saveubjson``.

-------------------------------------------------------------------------------

II. Installation

The installation of JSONLab is no different from installing any other
MATLAB toolbox. You only need to download/unzip the JSONLab package
to a folder, and add the folder's path to MATLAB/Octave's path list
by using the following command:

    addpath('/path/to/jsonlab');

If you want to add this path permanently, you need to type "pathtool", 
browse to the zmat root folder and add to the list, then click "Save".
Then, run "rehash" in MATLAB, and type "which savejson", if you see an 
output, that means JSONLab is installed for MATLAB/Octave.

If you use MATLAB in a shared environment such as a Linux server, the
best way to add path is to type 

   mkdir ~/matlab/
   nano ~/matlab/startup.m

and type addpath('/path/to/jsonlab') in this file, save and quit the editor.
MATLAB will execute this file every time it starts. For Octave, the file
you need to edit is ~/.octaverc , where "~" is your home directory.

=== Install JSONLab on Fedora 24 or later ===

JSONLab has been available as an official Fedora package since 2015. You may
install it directly using the below command

   sudo dnf install octave-jsonlab

To enable data compression/decompression, you are encouraged to install <tt>octave-zmat</tt> using

   sudo dnf install octave-zmat

=== Install JSONLab on Arch Linux ===

JSONLab is also available on Arch Linux. You may install it using the below command

   sudo pacman -S jsonlab

-------------------------------------------------------------------------------

III.Using JSONLab

JSONLab provides two functions, <tt>loadjson</tt> -- a MATLAB->JSON decoder, 
and <tt>savejson</tt> -- a MATLAB->JSON encoder, for the text-based JSON, and 
two equivallent function pairs -- <tt>loadubjson</tt> and <tt>saveubjson</tt> for binary 
JSON and <tt>loadmsgpack</tt> and <tt>savemsgpack</tt> for MessagePack. The <tt>load</tt> functions 
for the 3 supported data formats share almost the same input parameters; 
similarly for the 3 <tt>save</tt> functions (``savejson/saveubjson/savemsgpack``)
The detailed help information can be found in the <tt>Contents.m</tt> file. 

In the below section, we simply provide a few examples on how to use
each of the core functions for encoding/decoding JSON/UBJSON/MessagePack data

=== savejson.m ===

       jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... 
                'MeshElem',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],...
                'MeshSurf',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;...
                           2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],...
                'MeshCreator','FangQ','MeshTitle','T6 Cube',...
                'SpecialData',[nan, inf, -inf]);
       savejson(jsonmesh)
       savejson('jmesh',jsonmesh)
       savejson('',jsonmesh,'compact',1)
       savejson('jmesh',jsonmesh,'outputfile.json')
       savejson('',jsonmesh,'ArrayIndent',0,'FloatFormat','\t%.5g','FileName','outputfile2.json')
       savejson('cpxrand',eye(5)+1i*magic(5))
       savejson('ziparray',eye(10),'Compression','zlib','CompressArraySize',1)
       savejson('',jsonmesh,'ArrayToStruct',1)

=== loadjson.m ===

       loadjson('{}')
       dat=loadjson('{"obj":{"string":"value","array":[1,2,3]}}')
       dat=loadjson(['examples' filesep 'example1.json'])
       dat=loadjson(['examples' filesep 'example1.json'],'SimplifyCell',1)

=== saveubjson.m ===

       a={single(rand(2)), struct('va',1,'vb','string'), 1+2i};
       saveubjson(a)
       saveubjson('rootname',a,'testdata.ubj')
       saveubjson('zeros',zeros(100),'Compression','gzip')

=== loadubjson.m ===

       obj=struct('string','value','array',single([1 2 3]),'empty',[],'magic',uint8(magic(5)));
       ubjdata=saveubjson('obj',obj);
       dat=loadubjson(ubjdata)
       class(dat.obj.array)
       isequaln(obj,dat.obj)
       dat=loadubjson(saveubjson('',eye(10),'Compression','zlib','CompressArraySize',1))

=== jdataencode.m ===

      jd=jdataencode(struct('a',rand(5)+1i*rand(5),'b',[],'c',sparse(5,5)))
      savejson('',jd)

=== jdatadecode.m ===

      rawdata=struct('a',rand(5)+1i*rand(5),'b',[],'c',sparse(5,5));
      jd=jdataencode(rawdata)
      newjd=jdatadecode(jd)
      isequaln(newjd,rawdata)


=== examples ===

Under the "examples" folder, you can find several scripts to demonstrate the
basic utilities of JSONLab. Running the "demo_jsonlab_basic.m" script, you 
will see the conversions from MATLAB data structure to JSON text and backward.
In "jsonlab_selftest.m", we load complex JSON files downloaded from the Internet
and validate the loadjson/savejson functions for regression testing purposes.
Similarly, a "demo_ubjson_basic.m" script is provided to test the saveubjson
and loadubjson functions for various matlab data structures.

Please run these examples and understand how JSONLab works before you use
it to process your data.

-------------------------------------------------------------------------------

IV. Known Issues and TODOs

JSONLab has several known limitations. We are striving to make it more general
and robust. Hopefully in a few future releases, the limitations become less.

Here are the known issues:

# 3D or higher dimensional cell/struct-arrays will be converted to 2D arrays;
# When processing names containing multi-byte characters, Octave and MATLAB \
can give different field-names; you can use feature('DefaultCharacterSet','latin1') \
in MATLAB to get consistant results
# <tt>savejson</tt> can only export the properties from MATLAB classes, but not the methods
# saveubjson converts a logical array into a uint8 ([U]) array
# a special N-D array format, as defined in the JData specification, is implemented in \
<tt>saveubjson</tt>. You may use <tt>saveubjson(...,'NestArray',1)</tt> to create UBJSON \
Draft-12 compliant files 
# loadubjson can not parse all UBJSON Specification (Draft 9) compliant \
files, however, it can parse all UBJSON files produced by saveubjson.

-------------------------------------------------------------------------------

V. Contribution and feedback

JSONLab is an open-source project. This means you can not only use it and modify
it as you wish, but also you can contribute your changes back to JSONLab so
that everyone else can enjoy the improvement. For anyone who want to contribute,
please download JSONLab source code from its source code repositories by using the
following command:

      git clone https://github.com/fangq/jsonlab.git jsonlab

or browsing the github site at

      https://github.com/fangq/jsonlab

Please report any bugs or issues to the below URL:

      https://github.com/fangq/jsonlab/issues

Sometimes, you may find it is necessary to modify JSONLab to achieve your 
goals, or attempt to modify JSONLab functions to fix a bug that you have 
encountered. If you are happy with your changes and willing to share those
changes to the upstream author, you are recommended to create a pull-request
on github. 

To create a pull-request, you first need to "fork" jsonlab on Github by 
clicking on the "fork" button on top-right of jsonlab's github page. Once you forked
jsonlab to your own directory, you should then implement the changes in your
own fork. After thoroughly testing it and you are confident the modification 
is complete and effective, you can then click on the "New pull request" 
button, and on the left, select fangq/jsonlab as the "base". Then type
in the description of the changes. You are responsible to format the code
updates using the same convention (tab-width: 8, indentation: 4 spaces) as
the upstream code.

We appreciate any suggestions and feedbacks from you. Please use the following
mailing list to report any questions you may have regarding JSONLab:

      https://github.com/fangq/jsonlab/issues

(Subscription to the mailing list is needed in order to post messages).

-------------------------------------------------------------------------------

V.  Acknowledgement

This toolbox contains modified functions from the below toolboxes:

=== loadjson.m ===

The <tt>loadjson.m</tt> function was significantly modified from the earlier parsers 
(BSD 3-clause licensed) written by the below authors

* Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713
    created on 2009/11/02
* Franois Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393
    created on  2009/03/22
* Joel Feenstra:
    http://www.mathworks.com/matlabcentral/fileexchange/20565
    created on 2008/07/03


=== loadmsgpack.m ===

* Author: Bastian Bechtold
* URL: https://github.com/bastibe/matlab-msgpack/blob/master/parsemsgpack.m
* License: BSD 3-clause license

Copyright (c) 2014,2016 Bastian Bechtold
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, 
are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this 
  list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice, 
  this list of conditions and the following disclaimer in the documentation 
  and/or other materials provided with the distribution.

* Neither the name of the copyright holder nor the names of its contributors 
  may be used to endorse or promote products derived from this software without 
  specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

== zlibdecode.m, zlibencode.m, gzipencode.m, gzipdecode.m, base64encode.m, base64decode.m ==

* Author: Kota Yamaguchi
* URL:https://www.mathworks.com/matlabcentral/fileexchange/39526-byte-encoding-utilities
* License: BSD License, see below

Copyright (c) 2012, Kota Yamaguchi
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this
  list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice,
  this list of conditions and the following disclaimer in the documentation
  and/or other materials provided with the distribution

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
