Skip to content

Preliminary Feasibility Study

An Interactive Study and Translation System Based on Thich Nhat Hanh’s Teachings

1. Project Overview

This project aims to create a multipart system using large language models (LLMs) trained on Thich Nhat Hanh’s Teachings. The system will serve primarily Plum Village monastics, and secondarily, mindfulness practitioners, providing:

  • A query-based text search system to retrieve relevant teachings.

  • An interactive model for exploring Thich Nhat Hanh’s teachings through text-based dialogue.

  • A multilingual translation engine, initially supporting English, Vietnamese, and French, with potential to expand to other languages such as Mandarin, Spanish, Japanese, etc.

This project is designed as a foundational system, which can be expanded in the future to handle deeper conversational tasks and additional languages as technological advancements and resources become available.

2. Scope and Objectives

Primary Objectives:

  1. Text Query and Search (BERT-Based):
    • A system where users can input queries on mindfulness topics (e.g., “mindfulness and compassion”) and receive relevant resources (text, video, or audio excerpts).
    • Prioritize contextual understanding to ensure relevant and accurate search results from Thich Nhat Hanh’s teachings.
  2. Interactive Textual Exploration (GPT-3-Based):
    • An exploratory interactive tool for users to engage with Thich Nhat Hanh’s teachings in a conversational format.
    • GPT-3 will be fine-tuned to respond meaningfully to user input, guiding exploration of teachings.
  3. Multilingual Translation Engine (mBART-Based, Expanding to Multiple Languages):
    • A translation tool focusing initially on English, Vietnamese, and French.
    • Capacity to expand later into other languages such as Mandarin, Spanish, and others.
    • Emphasis on nuanced, Plum Village style language to ensure accurate and meaningful translation, especially for complex content.

3. Research Component: Model Evaluation

Given the rapid development of models in NLP, this project will include a research phase to evaluate and choose the most appropriate models for the three primary system components.

3.1 Model Research for Text Query and Search:

  • BERT (Bidirectional Encoder Representations from Transformers): BERT excels at understanding context within text, making it ideal for retrieving relevant passages based on search queries.
  • Evaluation: Review BERT variations, such as multilingual BERT, to assess their suitability for understanding context in a multilingual corpus of teachings.
  • Alternatives: Other Transformer-based models such as RoBERTa or Longformer will be evaluated for handling large datasets or longer documents.

3.2 Model Research for Interactive Dialogue:

  • GPT-3: The project will leverage GPT-3 for generating conversational responses based on user input. GPT-3 is powerful for generating coherent, meaningful interactions.
  • Evaluation: The research will assess DialoGPT (a dialogue-optimized version of GPT-2) as an alternative, especially for its focus on human-like conversations.
  • Factors to Consider: Trade-offs between response quality (GPT-3’s strength) and fine-tuning flexibility (DialoGPT’s ease of customization for dialogue).

3.3 Model Research for Translation Engine:

  • mBART (Multilingual BART): The research will focus on mBART due to its strength in handling complex contextual relationships, which is essential for translating spiritual texts.
  • Evaluation: Compare mBART with MarianMT, which offers efficiency and lower computational demands. mBART’s superior handling of nuanced language will be balanced against the resource-efficient MarianMT, which is specifically designed for translation tasks.
  • Initial Language Focus: Fine-tune the translation engine for English, Vietnamese, and French, with a plan to expand to other languages as needed.

4. Technical Feasibility

4.1 Data Collection and Preparation

  • Text Corpus: Collect, preprocess, and check for accuracy, Thich Nhat Hanh’s writings (books, articles, Dharma talks, interviews) in English, Vietnamese, and French. The initial training set will be equivalent to approximately 100 books, sourced from talks, interviews, and books across the three languages, with the majority of the volume in Vietnamese. This dataset will serve as the foundation for query, translation, and interactive systems.
  • Audio/Visual Corpus: Include video/audio resources with transcriptions to support multimedia queries. Audio-to-text conversion models may be considered in the future.
  • Translation Data: Collect parallel texts for English-Vietnamese and English-French pairs to fine-tune the translation engine. This dataset will serve as the foundation for expanding into additional languages.

4.2 Model Fine-Tuning and Deployment

  • Cloud Infrastructure: Use cloud-based services like Google Cloud or AWS to fine-tune models and ensure scalability. The GPT-3 system can be trained through the Open AI API interface. Pre-trained models significantly reduce training time and resources, but fine-tuning will still require access to GPUs/TPUs.
  • Deployment: Hugging Face provides tools and APIs for easy model deployment. This infrastructure will be used for both training and serving models.

4.3 Computational Resources

  • Fine-tuning pre-trained models like BERT, and mBART will require significant computational resources, but this is mitigated by using pre-trained models as a foundation. GPT-3 can be trained entirely on the Open AI platform.
  • Ongoing resource needs will depend on the scalability of translation tasks and the number of supported languages.

5. Cost Feasibility

5.1 Initial Costs:

  • Fine-tuning pre-trained models and deploying them on cloud infrastructure.
  • Data preprocessing, audio to text conversion, and cleaning of the text corpus.
  • Training GPT-3 on the initial dataset (equivalent to 100 books) will cost approximately \$300. This does not include other Open AI platform costs, but provides a ballpark figure for the initial training phase.
  • Translation engine setup for initial languages (English, Vietnamese, French).

5.2 Ongoing Costs:

  • Hosting the models, ongoing training as the dataset grows, and maintaining cloud infrastructure for serving requests.
  • Expanding the translation system to include additional languages as needed.

5.3 Mitigation:

  • Starting with smaller datasets and key languages (English, Vietnamese, French) reduces initial costs and allows for gradual scaling.

6. Operational Feasibility

6.1 User Interface and Experience:

  • Develop a user-friendly interface allowing monastics and practitioners to search, interact, and request translations of texts.
  • Ensure accessibility across various devices (web and mobile) to facilitate widespread use among the Plum Village community and beyond.

6.2 Testing and Feedback:

  • Conduct pilot testing with a small group of Plum Village monastics to refine the system.
  • Collect feedback to fine-tune the query system, conversational responses, and translation accuracy.

7. Future Scalability and Expansion

  1. Language Expansion: As new bilingual data becomes available, expand the translation system to support languages like Mandarin, Spanish, Japanese, and Korean.
  2. Advanced Conversational Capabilities: Explore future opportunities to refine the interactive system, potentially scaling toward a more advanced conversational model that reflects deeper spiritual dialogues.
  3. Multimedia Integration: Add real-time transcription of audio content and expand the multimedia query engine to handle video/audio teachings.
  1. Data Ownership: Ensure permissions are secured for digitizing and using Thich Nhat Hanh’s teachings.
  2. Ethical AI: Implement guidelines to prevent misrepresentation or biased interpretations of the teachings. Ensure transparency in how models handle sensitive spiritual content.

Conclusion

The project is technically feasible using pre-trained models such as BERT, GPT-3, and mBART, with a strong focus on accuracy for translations and queries. It can scale over time to include additional languages and advanced functionalities. Starting with smaller, well-defined goals (English, Vietnamese, French translations; query and interactive systems) ensures cost-effective deployment and lays the groundwork for future expansion.