Content moderation using machine learning: the server-side part

• Read original article here

From here, you can select your project in the list, connect Firebase to an existing Google Cloud project, or create a new Firebase project.Once the project is configured, use the following command to deploy your Cloud Function:

Realtime Database in the Firebase console Cloud Functions logs Once deployment is complete, the logs include the link to your hosted guestbook. Write some guestbook entries. If you followed part 1 of the blog, you will need to either delete the moderation code from the website and deploy again, or manually add guestbook entries to the Realtime Database in the Firebase console You can view your Cloud Functions logs in the Firebase console. I have a bunch of ideas for ways to build on this example. Here are just a few. Let me know which ideas you would like to see me build, and share your suggestions as well! The best ideas come from collaboration.

I mentioned that the "match" value of a language label can be , or without going into detail on the significance of the  value. If the label is , then the model cannot determine if the language is toxic within the given threshold. One way to limit the number of  values is to lower this threshold. For example, if you change the threshold value to 0.8, then the model will label the match value as  if it is at least 80% certain that the text contains language that fits the label. My website example assigns labels of value  the same as those labeled , allowing that text through the filter. But since the model isn't sure if that text is appropriate, it's probably a good idea to get some eyes on it. You could add these posts to a queue for review, and then approve or deny them as needed. I said "you" here, but I guess I mean "me". If you think this would be an interesting use case to explore, let me know! I'm happy to write about it if it would be useful.

One of the things I like about the original Firebase moderation sample is that it sanitizes the text rather than just deleting the post. You could run text through the sanitizer before checking for toxic language through the text toxicity model. If the sanitized text is deemed appropriate, then it could overwrite the original text. If it still doesn't meet the standards of decent discourse, then you could still delete it. This might save some posts from otherwise being deleted.You've probably noticed that my moderation functionality doesn't extend to the name field. This means that even a halfway-clever troll could easily get around the filter by cramming all of their expletives into that name field. That's a good point and I trust that you will use some type of moderation on all fields that users interact with. Perhaps you use an authentication method to identify users so they aren't provided a field for their name. Anyway, you get it: I didn't add moderation to the name field, but in a production environment, you definitely want moderation on all fields.When you test out real-world text samples on your website, you might find that the text toxicity classifier model doesn't quite fit your needs. Since each social space is unique, there will be specific language that you are looking to include and exclude. You can address these needs by training the model on new data that you provide.If you enjoyed this article and would like to learn more about TensorFlow.js, then there are a ton of things you can do: