README

HanLP: Han Language Processing


    
       
    
    
        
    
    
        
    
    
        
    
    
        
    



    中文 |
    日本語 |
    Docs |
    Forum


HanLP is the multilingual NLP library designed for researchers and enterprises, built on PyTorch and TensorFlow 2.x to advance state-of-the-art deep learning techniques in academia and industry. HanLP was designed from day one to be
efficient, user-friendly and extendable.

Thanks to open-access corpora like Universal Dependencies and OntoNotes, HanLP 2.1 now offers 10 joint tasks on 130
languages: tokenization, lemmatization, part-of-speech tagging, token feature extraction, dependency parsing,
constituency parsing, semantic role labeling, semantic dependency parsing, abstract meaning representation (AMR)
parsing.

For end users, HanLP offers light-weighted RESTful APIs and native Python APIs.

 RESTful APIs

Tiny packages in several KBs for agile development and mobile applications. Although anonymous users are welcomed, an
auth key is suggested
and a free one can be applied here under
the CC BY-NC-SA 4.0 license.


  Click to expand tutorials for RESTful APIs

   Python

  bash
  pip install hanlp_restful
  

  Create a client with our API endpoint and your auth.

  python
  from hanlp_restful import HanLPClient
  HanLP = HanLPClient('https://hanlp.hankcs.com/api', auth=None, language='mul')  Support en, ja, zh, mul
  

   Java

  Insert the following dependency into your pom.xml.

  xml
  
    com.hankcs.hanlp.restful
    hanlp-restful
    0.0.15
  
  

  Create a client with our API endpoint and your auth.

  java
  HanLPClient HanLP = new HanLPClient("https://hanlp.hankcs.com/api", null, "mul"); // Support en, ja, zh, mul
  

   Quick Start

  No matter which language you use, the same interface can be used to parse a document.

  python
  HanLP.parse(
      "In 2021, HanLPv2.1 delivers state-of-the-art multilingual NLP techniques to production environments. 2021年、HanLPv2.1は次世代の最先端多言語NLP技術を本番環境に導入します。2021年 HanLPv2.1为生产环境带来次世代最先进的多语种NLP技术。")
  

  See docs for visualization, annotation guidelines and more details.




 Native APIs

bash
pip install hanlp


HanLP requires Python 3.6 or higher. While GPU or TPU acceleration is recommended, it is not mandatory.

 Quick Start

python
import hanlp

HanLP = hanlp.load(hanlp.pretrained.mtl.UD_ONTONOTES_TOK_POS_LEM_FEA_NER_SRL_DEP_SDP_CON_XLMR_BASE)
print(HanLP(['In 2021, HanLPv2.1 delivers state-of-the-art multilingual NLP techniques to production environments.',
             '2021年、HanLPv2.1は次世代の最先端多言語NLP技術を本番環境に導入します。',
             '2021年 HanLPv2.1为生产环境带来次世代最先进的多语种NLP技术。']))


- In particular, the Python HanLPClient can also be used as a callable function following the same semantics.
  See docs for visualization, annotation guidelines and more details.
- To process English, Chinese or Japanese, HanLP provides mono-lingual models in each language which significantly outperform the
  multilingual model. See docs for the list of

Top issues

Latest release

After supporting 130 languages, HanLP has officially released an open-source Ancient Chinese model. This model supports automatic word segmentation, lemmatization, part-of-speech tagging, and dependency parsing for Ancient Chinese. Thanks to multi-task learning, this single model can handle all of these tasks, as well as coarse-grained/fine-grained segmentation and UPOS/XPOS/PKU part-of-speech tagging sets.

- Blog post: https://www.hankcs.com/nlp/hanlp-ancient-chinese-processing-model-released.html
- Demo: https://github.com/hankcs/HanLP/blob/master/plugins/hanlp_demo/hanlp_demo/lzh/demo_m