⚙️ Engineering Standards

This document outlines the coding and collaboration standards we'll follow to ensure our project is consistent, maintainable, and easy to work on.

🌳 Git & GitHub Workflow

• One Feature, One Pull Request (PR): Each pull request should focus on a single, specific feature. Avoid combining multiple, unrelated changes into a single PR.
• Descriptive Branch Names: Make the branch name descriptive. Eg: feat/train-scripts. Don’t have meaningless branch names.
• Commit Message Conventions: Follow proper git commit message conventions: https://www.conventionalcommits.org/en/v1.0.0/
• Branching Strategy:
  • Before creating a new branch, please ask the project lead on which parent branch you should branch out from.
  • We will try to only maintain one or two main branches. Until the project/number of collaborators grows large.
  • Try to keep a clean + linear commit history. What I mean by that is try to branch out from main and merge to main as soon as the feature is implemented. All branches that are merged should be deleted after the merge. All branches should be squashed and merged (this could be set as the default in GitHub settings).
  • Avoid second/third-order branches—they just cause confusion. If you feel the need to, consult the project lead.
  • Whenever you create a new branch, create a draft PR immediately. Convert that into a normal PR when you are ready to merge. Always ask for a review from the project lead before merging.
  • When reviewing PRs, take the time to actually review all the changes meaningfully and ideally also running this tests for the changes locally before approving the PR.
• Protecting the main Branch:
  • Never push to the main branch directly. I’ll be protecting the main branch from doing so. Changes will only be made to the main branch through a reviewed and approved PR.
  • Any change made to main needs to be stable. This means that all existing tests must pass. If not, first fix the tests in the feature branch and once they pass, only then is the PR merged.

💾 Data & File Management

• No Data Files: No data file is to be committed. Non-data files that exceed > 5 MB should be consulted before committing.
• Language Preference: For the purpose of our projects, stick to using Python scripts wherever you can. Only use other languages such as a bash script if it's obviously beneficial or there was no other simpler option.

💻 Code Style & Conventions

• Atomic Commits: Always ensure that a single git commit reflects a single meaningful change. This doesn’t mean that every file needs to be committed separately, but if you do two different changes that are sort of independent, then both need to be committed separately.
  • Example: If I updated my .gitignore to add something like .code-workspace to it along with adding a new function to a separate script, then both changes should be committed separately. If, on the other hand, you renamed a file somewhere and thus to import it, you also changed the name in the entire codebase, all those changes should be committed together.
• Tests for Everything: Write tests for everything. I mean literally everything. Test that the folders you are creating are created properly, all the files are saved properly. Handle as many edge cases as you can think of. Add a comment on the top of each test denoting your philosophy of writing that tests. comments can be incredibly helpful.

📝 Documentation: Comments & Docstrings

• Function Docstring Format: The format of a function docstring should be as follows. This should include a description of the function. Don’t have one single long line. If the description is long, split into multiple lines.

  def create_folder_structure(
      base_data_dir: Path, total_files: int, files_per_folder: int = 1000
  ) -> Dict[int, Path]:
      """Create numbered folders for organizing structure files.
  
      Args:
          base_data_dir (Path) : Base directory
          total_files (int) : Total number of files to process
          files_per_folder (int) : Number of files per subfolder. 
  	        Defaults to 1000
  
      Returns:
          dict: Dictionary mapping folder numbers to folder paths
      """
      unzipped_dir = base_data_dir / "cif_unzipped"
      unzipped_dir.mkdir(exist_ok=True)
  

• Tensor Shapes: With tensors, it’s also wise to specify the Expected shape.

  def concat_tensors(tensor_1: torch.Tensor, tensor_2: torch.Tensor) -> torch.Tensor:
  	"""
  	Concats two tensors
  	
  	Args:
  		tensor_1 (torch.Tensor): First tensor. 
  			Expected Shape: batch_size * sequence_length * embedding_size
  	"""
  	concat_tensor = torch.cat(tensor_1, tensor_2)
  	return concat_tensor