Lingosta: Discover an AI-powered personalized language-learning journey

Introduction

Being connected with a diverse network of people from Brazil on social media has sparked a strong desire within me to read and write in Brazilian Portuguese. I wanted to communicate casually with my friends and online contacts from Brazil in their native language.

For my particular use case, I found traditional language learning methods to be overwhelming, as they focused on excessive vocabulary and impractical phrases that I would rarely use. For instance, one of the most widely used language-learning apps in the market continuously prompts me to practice the sentence, "The shark drinks lemonade." I longed for a simpler and more personalized approach, where I could learn and practice the language with relevant content that aligned with my own language goals.

My previous experience working with the Appwrite product and its team had been overwhelmingly positive so when Hashnode announced the Appwrite Hackathon, I saw the perfect opportunity to build a language-learning platform. This platform would enable people to learn and practice another language with content that holds true relevance to their everyday lives.

Introducing Lingosta: the language learning app that revolutionizes non-verbal learning. With its AI-powered features, Lingosta instantly translates sentences into flashcards. The AI engine also curates exercises of custom sentences using your translated words. Embrace a seamless and true personalized learning experience with Lingosta.

Overview

Lingosta's main philosophy is to enable you to learn vocabulary that is relevant to your own language-learning journey.

It accomplishes this with 3 main features:

• Translations - Lingosta will translate a sentence into English and store the translations

• Flash Cards - Saved translations are automatically generated into flash cards that you can use right away to practice

• Mix and Match (Jumbles) - Lingosta runs a scheduled process that will inspect a user's translated words and use AI to generate sentences. These sentences are then used in a Mix and Match exercise named Jumbles.

These are all namespaced into Lingosta Groups. Groups are named and tied to a language. Lingosta supports Spanish, Portuguese, Korean, French, German, Italian, Dutch, Russian, Chinese, and Japanese.

Demo

Below is a video of a live walkthrough of Lingosta!

Technologies

• NextJS

  • ReactJS - versatile JavaScript library with a thriving ecosystem of libraries and tools

  • Tailwind CSS - Styling framework for creating responsive interfaces, enabling a web and mobile interface

  • NodeJS - Backend technology used to create API endpoints for translations

• Appwrite Cloud

  • Authentication - Handles the authentication workflow and persists user settings

  • Database - Manages the persistence layer for translations and exercises

  • Cloud Function - Executes a daily scheduled process to generate customized sentences using translations

• Appwrite CLI - Runs bootstrap scripts for creating collections

• Vercel - Deployment platform offering a smooth development lifecycle for NextJS applications

• OpenAI API - External service utilized for translations and processing words into sentences

Quick Thoughts About Appwrite

Appwrite provided a pleasant and empowering experience, allowing us to work efficiently and achieve optimal progress. It offers a focused feature set without unnecessary complexity, and its documentation is concise and effective, enabling us to quickly become productive. The vibrant GitHub threads and active Discord community provided invaluable support, reinforcing our decision to continue to invest in integrating Appwrite into Lingosta beyond the beta phase.

While our usage of authentication and database persistence is relatively straightforward, I want to emphasize a unique feature of Appwrite's Authentication: the ability to store user preferences. In Lingosta, we utilize "Groups" as namespaces for organizing translations in different languages. Whenever a user switches their active group, we persist it in their preferences, ensuring that they see their selected group upon logging in. This simple and effective feature further reinforces Appwrite’s developer-centric focus.

Challenges

During development, we ran into 2 main challenges: the portability of Appwrite Database collections and Appwrite Database’s lack of objects as schema attributes.

Portability

The Challenge

Ensuring version control and portability of our database schemas was important for our project. Version control enables us to track the evolution of our schemas, keeping them in sync with our application code. Portability allows us to seamlessly deploy our schemas across multiple environments, ensuring consistency throughout.

Now, when utilizing the self-hosted version of Appwrite, you can interface with the MariaDB layer for tasks such as managing database migrations and backups. However, in the case of Appwrite Cloud, the underlying database is not directly exposed or accessible to the user.

Our Solution

To handle this, we used Appwrite CLI to set up a series of migration scripts to bootstrap the Appwrite collections. For Lingosta, we have 2 projects: “Lingosta Staging” for testing and “Lingosta Production” for the live site. The collections in “Lingosta Production” were created effortlessly using these bootstrap scripts.

Solution Bonus: Accurate Metrics

The Appwrite Console provides a range of dashboards, allowing us to effectively monitor user activity and gain valuable insights into our metrics. It’s crucial that a live application has metrics and our setup offers the advantage of a clear separation for our Appwrite dashboards.

The screenshot below shows one of the many dashboards available in the Appwrite Console. Having a production environment distinct from our test environment provides us with more accurate metrics on user sign-ups and activity. We plan on referencing these dashboards a lot to observe additional traffic that we anticipate from Lingosta's beta launch with the release of this article!

Schema Attribute Limitations

The Challenge

Since our application was written in TypeScript, we wanted to model our schemas with JSON. With JSON, it’s conventional to have nested objects to represent the data that you’re exposing. While Appwrite supports a rich set of primitives for attributes, it doesn’t support objects.

The prevalent workaround was to convert JSON objects into string format and store them in string fields, which felt incredibly sloppy. In Appwrite, a string attribute is defined along with the expected length of the string data. Storing stringified JSON objects required setting an arbitrarily long string length in our attributes. In addition, storing objects as strings also result in the loss of server-side query capabilities. Our goal was to steer clear of this approach and aimed to find solutions that would allow us to work with our data effectively, without feeling constrained or limited by the platform.

Our Solution

We established a module that clearly defines our data models through three interfaces: one for representing the data in Appwrite, one for representing user-defined fields, and another for representing it in our application. Mappers were implemented for each interface to handle the serialization and deserialization processes. This allows our data to have a distinct representation for storage in Appwrite, an intermediary format where we can hydrate data for Appwrite-managed fields (such as $id and $createdAt), and another representation tailored to our application's needs.

Now, let's walk through this setup process by using a simplified representation of how we store translations.

1. UserTranslation - This is the object model that we use across our application. As far as our React and NodeJS applications are concerned, this is the only object model that we use for handling translations. The terms field is a nested object implementing the Terms interface.

   Example:

    {
        ownerId: "owner_id",
        sourceLanguage: "pt-br", // Brazilian Portuguese
        terms: [
            {
              source: "eu",
              target: "I",
              weight: 0.9
            },
            {
              source: "amo",
              target: "love",
              weight: 0.5          
            },
            {
              source: "Appwrite",
              target: "Appwrite",
              weight: 0.1     
            },
        ],
        // ...other fields
        id: "document_id",
        createdAt: "2023-06-13T08:30:00.000Z"
    }
   

2. DBTranslationFields - This type represents the data that we will want to save into Appwrite. You'll notice that the terms field has been deconstructed into individual arrays. This allows us to store the terms using the Appwrite primitives and avoid stringified JSON fields as we mentioned previously.

   It's used as an intermediary in the wrapper and it's composed purely of application-specific fields for setting data.

   Example:

    {
        ownerId: "owner_id",
        source_translations: ["eu", "amo", "Appwrite"],
        target_translations: ["I", "love", "Appwrite"],
        translation_weights: [0.9, 0.5, 0.1]
        sourceLanguage: "pt-br", // Brazilian Portuguese
        // ...other fields
    }
   

   This type is important for 2 reasons:

   1. We can't write to Appwrite with the DBTranslation because it contains managed fields (which we will cover shortly)

   2. Typescript complains when we try to define objects with just DBTranslation because they don't implement all of the Appwrite Models.Document fields

3. DBTranslation - This object represents a 1:1 mapping of how data is stored in Appwrite because it implements Appwrite's Models.Document.

   Notice the $id and $createdAt values in the JSON object in the example. These fields are automatically managed by Appwrite, and if you attempt to write or update the document with these fields included, Appwrite will raise an error. To address this, DBTranslationFields is employed to eliminate these fields from the object before interacting with the Appwrite database.

   Example:

    {
        ownerId: "owner_id",
        source_translations: ["eu", "amo", "Appwrite"],
        target_translations: ["I", "love", "Appwrite"],
        translation_weights: [0.9, 0.5, 0.1]
        sourceLanguage: "pt-br", // Brazilian Portuguese
        // ...other fields
        $id: "document_id"
        $createdAt: "2023-06-13T08:30:00.000Z"
    }
   

We use a mapper to deserialize data from Appwrite and serialize the data to write into Appwrite.

While this approach may appear complex, it is a common practice to have distinct representations of schemas and utilize mappers to handle them. We implemented this setup for all our Appwrite collections. Although there was initial overhead involved in setting it up, the investment proved worthwhile almost immediately.

• By implementing centralized models, we significantly minimized the occurrence of data model inconsistencies across our codebase. This ensured that our frontend, backend, and database schemas remained consistent, reducing potential issues related to schema synchronization.

• Through definition of our models using TypeScript, we reaped the advantages of type safety, mitigating bugs associated with accessing undefined properties. This ensured a more robust and error-free codebase, contributing to a smoother development process.

• With the establishment of an abstraction layer for representing data in Appwrite, we achieved a clear separation of concerns, ensuring that our pages and components remained free from any Appwrite-specific logic. This architectural approach enhanced maintainability and modularity within our codebase.

Next Steps

Lingosta was designed with a long-term vision, extending beyond the duration of this Hackathon. We firmly believe that there is a viable market for this product, driven by our confidence in its value proposition and potential impact. Appwrite has convinced us that it's a compelling platform and that we should continue to invest in its usage.

To achieve these goals:

• We have diligently built a maintainable codebase, ensuring scalability and ease of future development.

• Our application boasts a responsive design that functions seamlessly on mobile devices.

• We have prioritized search engine optimization (SEO) for our landing page, maximizing its visibility and reach.

• We take pride in our exceptional Lighthouse score, reflecting our commitment to delivering a performant and optimized user experience.

  

Learn With Lingosta!

Lingosta is live! Please feel free to visit Lingosta and create an account at https://lingosta.app!

Please note that:

• As our team possesses a certain level of proficiency in reading Spanish, we have been conducting tests with Spanish translations within Lingosta.

• Currently, our daily Jumbles generation function is temporarily disabled due to cost considerations. Instead, we have implemented a static Jumbles exercise in Spanish to assess user engagement. We are considering the possibility of offering this as a paid feature in our application.

The code is open source and can be found on our GitHub repository.

Thank You!

Special thanks to Hashnode and Appwrite for the opportunity to create and share Lingosta.

Thank you for taking the time to learn about Lingosta!

-- Justin Lee and Nam Le