User Study Self-Report Bias
The user study relies on participants' self-reported satisfaction, which may be subject to a positive response bias and not fully capture objective platform quality or identify subtle areas for improvement.
Potential for Personal Filter Bubbles
While aiming to mitigate filter bubbles from social-factor-based recommendations, Scholar Inbox's reliance on explicit positive and negative user ratings for its content-based model could still inadvertently create personalized filter bubbles, narrowing a user's exposure to diverse research over time, a known limitation of content-based systems.
Limited Explicit Modeling of Diverse Interests
The paper acknowledges that a common criticism from its user study is the platform's current inability to explicitly model multiple, distinct research interests for a single user, suggesting that researchers with highly diverse needs might not receive optimal recommendations across all their areas.
Generalizability of Satisfaction Data
While the platform caters to diverse scientific fields, the detailed satisfaction data presented in the user study (Figure 6) specifically highlights strong results in Machine Learning, Computer Vision, and Robotics. This might limit the generalizability of these high satisfaction levels to all scientific disciplines using the platform.