Add link to code repository

README.md

@@ -1,20 +1,23 @@
 ---
-license: mit
-pipeline_tag: image-text-to-text
-library_name: transformers
 base_model:
 - internlm/internlm2-chat-1_8b
-base_model_relation: merge
 language:
 - multilingual
+library_name: transformers
+license: mit
+pipeline_tag: image-text-to-text
 tags:
 - internvl
 - vision-language model
 - monolithic
+base_model_relation: merge
 ---
+
 # HoVLE
 
-[\[📜 HoVLE Paper\]](https://arxiv.org/pdf/2412.16158) [\[🚀 Quick Start\]](#quick-start)
+[\\[📜 HoVLE Paper\\]](https://arxiv.org/pdf/2412.16158) [\\[🚀 Quick Start\\]](#quick-start)
+
+Code: https://github.com/OpenGVLab/HoVLE
 
 <a id="radar"></a>
 
@@ -40,12 +43,11 @@ This repository releases the HoVLE model with 2.6B parameters. It is built upon
 |                               | Details | 
 | :---------------------------: | :---------- |
 |          Architecture         |  The whole model consists of a holistic embedding module and an LLM. The holistic embedding module consists of the same causal Transformer layers as the LLM. It accepts both images and texts as input, and projects them into a unified embedding space. These embeddings are then forwarded into the LLM, constituting a monolithic VLM.  |
-|     Stage I (Distillation)    |  The first stage trains the holistic embedding module to distill the image feature from a pre-trained visual encoder and the text embeddings from an LLM, providing general encoding abilities. Only the holistic embedding module is trainable. |
+|     Stage I (Distillation)    |  The first stage trains the holistic embedding module to distill the image feature from a pre-trained visual encoder and the text embeddings from the LLM, providing general encoding abilities. Only the holistic embedding module is trainable. |
 |     Stage II (Alignment)      |  The second stage combines the holistic embedding module with the LLM to perform auto-regressive training, aligning different modalities to a shared embedding space. Only the holistic embedding module is trainable. |
 |     Stage III (Instruction Tuning)   |  A visual instruction tuning stage is incorporated to further strengthen the whole VLM to follow instructions. The whole model is trainable. |
 
 
-
 ## Performance
 <p align="middle">
   <img src="assets/performance1.png" width="90%" />
@@ -58,11 +60,9 @@ This repository releases the HoVLE model with 2.6B parameters. It is built upon
 - Please note that evaluating the same model using different testing toolkits can result in slight differences, which is normal. Updates to code versions and variations in environment and hardware can also cause minor discrepancies in results.
 
 
-
 Limitations: Although we have made efforts to ensure the safety of the model during the training process and to encourage the model to generate text that complies with ethical and legal requirements, the model may still produce unexpected outputs due to its size and probabilistic generation paradigm. For example, the generated responses may contain biases, discrimination, or other harmful content. Please do not propagate such content. We are not responsible for any consequences resulting from the dissemination of harmful information.
 
 
-
 ## Quick Start
 
 We provide an example code to run HoVLE inference using `transformers`.
@@ -105,7 +105,7 @@ def find_closest_aspect_ratio(aspect_ratio, target_ratios, width, height, image_
             best_ratio_diff = ratio_diff
             best_ratio = ratio
         elif ratio_diff == best_ratio_diff:
-            if area > 0.5 * image_size * image_size * ratio[0] * ratio[1]:
+            if area > 0.5 * image_size * ratio[0] * ratio[1]:
                 best_ratio = ratio
     return best_ratio
 
@@ -190,10 +190,8 @@ print(f'User: {question}\nAssistant: {response}')
 question = 'Please write a poem according to the image.'
 response, history = model.chat(tokenizer, pixel_values, question, generation_config, history=history, return_history=True)
 print(f'User: {question}\nAssistant: {response}')
-
 ```
 
-
 ## License
 
 This project is released under the MIT license, while InternLM2 is licensed under the Apache-2.0 license.