Steam Review aspect dataset is a dataset of Steam Review with 8 review aspects. This dataset contains 1100 English Steam reviews, split into 900 train and 200 test. This dataset was initially created to identify which aspects are mentioned in English reviews as part of Analysis of 43 million English Steam Reviews.

Table of contents

• Data collection and annotation
• Data format
• Model benchmark
• Download
• Citation
• License
• Appendix A - Statistic
• Appendix B - Full model benchmark

Data collection and annotation

The source of the reviews comes from a snapshot of the SRec database, which was taken on 21 February 2024. SRec obtain all reviews for all games and mods using API provided by Steam. To reduce bias when selecting reviews to be annotated, I chose reviews primarily based on these criteria,

• Character length.
• Helpfulness score.
• Popularity of the reviewed game.
• Genre or category of the reviewed game.

There are 8 aspects to define review in this dataset. I am the only annotator for this dataset. A review is deemed to contain a certain aspect, even if it's mentioned implicitly (e.g., "but it'd be great if there's good looking characters...") or only mention lack of the aspect (e.g., "... essentially has no story ..."). The below table shows 8 aspects of this dataset, along with a short description and example.

Take note that few reviews contain language and content that some people may find offensive, discriminatory, or inappropriate. I DO NOT endorse, condone or promote any of such language and content.

Data format

CSV, JSON and Apache Arrow file formats are provided for convenience's sake. You can check the notebook on example directory for a bare-minimum example of how to open those files. Both raw and cleaned review text are provided. Cleaned review text was preprocessed by stripping BBcode, reducing excessive whitespaces and reducing excessive newlines.

Model benchmark

Model benchmark on Steam Review aspect dataset split into 3 categories,

• Base: Non-attention based language model.
• Embedding: Inspired by MTEB, obtained embedding trained on Logistic Regressor for up to 100 epochs.
• Fine-tune.

There are 15 models benchmarked, where few of the same base models are used multiple times using different methods. You can see Appendix B for the result and visit GitHub to see its source code.

Download

You can download Steam review aspect dataset from one of these sources,

• GitHub
• Huggingface
• Kaggle

Citation

If you wish to use this dataset in your research or project, please cite this blog post: Steam review aspect dataset

Sandy Khosasi. "Steam review aspect dataset". (2024).

For those who need it, a BibTeX citation format also has been prepared.

@misc{srec:steam-review-aspect-dataset,
	title        = {Steam review aspect dataset},
	author       = {Sandy Khosasi},
	year         = {2024},
	month        = {may},
	day          = {28},
	url          = {https://srec.ai/blog/steam-review-aspect-dataset},
    urldate      = {2024-05-28}
}

License

Steam Review aspect dataset is licensed under Creative Commons Attribution 4.0 International.

Appendix A - Statistic

Appendix B - Full model benchmark