By Ricardo Lezama 
Yesterday, the Mexican NLP Summer School ushered in the second day of enlightening presentations, tutorials and knowledge sharing from experts globally in the context of NAACL 2024 (North American Association of Computational Linguistics). The event preempts the commencement of the much-anticipated NAACL 2024 conference set to take place in Mexico City.
The two-day summer school was sponsored in part by private enterprise, like Grammarly and Bloomberg LP, while the organizing aspect was managed by researchers Thamar Solorio from Mohamed bin Zayed University of Artificial Intelligence, Ximena Gutierrez-Vasques and other members of the UNAM and Mexico City’s vast network of research faculty as well as student volunteers from abroad and within Mexico. Notably, Helena Gomez, a researcher and professor from UNAM, coordinated vast amounts of the events as both attested by the website for NAACL and on the ground participants.
Points Of Contact Between Global Industry, Research Outside of Global North & LATAM
Some of the interest from private industry and international universities reflects both a belief in the vast network of talent in Mexico City as well as shared problems. While large language models are both a puzzling and effective means towards greater applications of NLP and conversational AI, they are also difficult to explain and too crude to be applied with elegance towards more sophisticated settings. The probabilistic nature of output makes for the possibility of erroneous output given a conversational context, known as ‘hallucinations’.
In the case of Bloomberg, they have internally tested their own GPT model for finance applications with a paper published on the matter earlier this year. The model at the time of launch was the largest domain specific model in the world, concentrating in the finance domain, with refinements taking place internally.
BBGPT is built on a massive 345 billion token dataset, which is the largest domain-specific dataset ever, & This LM w/ 50 billion parameter will be assisting BB in improving their existing financial NLP tasks. pic.twitter.com/QNp7CXgvCO
— Mayur (@machopraa) April 5, 2023
Meanwhile, MBZAI’s university has built their own Large Language Models to execute applications in the Conversational AI space. The university has claimed the model outperforms Llama-2.
The spirit of both projects resonates well with presenters who seek to not necessarily discredit the hype around LLM’s, but to make strides in more specific applications with their considerations taking center stage. Particularly, the call to make strides outside of the corporate-academic nexus between American and European Industry and their regional powerhouse universities.
#UAE’s Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) has launched the K2-65B large language model (LLM), claiming it outperforms #Meta’s Llama 2 model. #Forbes
— Forbes Middle East (@Forbes_MENA_) May 30, 2024
For more details: 🔗https://t.co/PHHumghGWu pic.twitter.com/wTihiasXkX
Highlights
A wide variety of research topics have been set to unravel in this conference. Ranging from zero-shot hierarchical table analysis using Augmented Language Models, presentations on fairness, safety, and multilinguality to discussions on biomedical entities in low-resource settings, the conference portfolio promises to offer an inspiring agglomeration of the latest AI and NLP research.
Forums to explain the need for grounded research in LLM applications was highlighted in a panel discussion conducted by both academic and industry experts based in Mexico City or somehow connected to regional network. Specifically, the need for regional experts to define goals beyond the usual metrics linked to performance and define projects that meet human needs.
Yesterday, June 14th, the panel "Perspectives on NLP from Latin America" took place at @EscuelaPLNmex, featuring @LucianaBenotti alongside @jocelyndunstane, Aide Paola Rios Rivero, and Helena Gomez.
— Fundación Vía Libre (@FViaLibre) June 15, 2024
It was amazing! Thank you for the invitation.#NAACL2024 #MexicanNLPSummerSchool pic.twitter.com/0yViN1fwFs
One of the most comprehensive tutorials was given by Danae Sánchez who reviewed the first Bag of Word methods all the way through to LLM’s most modern incarnation, as well as the grounbreaking research conducted by her team at the University of Copenhagen. Using the PIXELS model, the team has managed to skip over the traditional NLP pipeline and train language models as if they were vision models. Surely, this aforementioned description does not do the team justice. Nonetheless, for now and to capture the moment, we leave her tweet below:
Arriving at the Mexican NLP Summer School @EscuelaPLNmex I'll be presenting a tutorial "Exploring Transformers and Limitations in Language Modeling" at 2 pm #NLP pic.twitter.com/sAtkUs0f0S
— Danae Sánchez (@danaesavi) June 15, 2024
From the Hilton in Mexico City, the NAACL conference presents an incredible opportunity for scholars to showcase their original research. Among these distinguished contributors, attendees were eager to present papers that tackled various subjects. One of the highlights, “E5: Zero-shot Hierarchical Table Analysis using Augmented LLMs via Explain, Extract, Execute, Exhibit, and Extrapolate”, while the full program is available online.