Relatively small models when fine-tuned to a specific domain and task can out perform much larger, general models. The below examples compare answers between two models: a Falcon-7b trained on 51k examples taken from US Army publications, and GPT-3.5 (text-davinci-003). Both models are in the same RAG stack: llama-index, Chroma-DB vector DB of Army pubs, and basic prompt (“from the following context, and only using the context, answer the question. Do not use prior knowledge”). Temp was set to 0.6. Takeaways:
- They seem to perform equally well at simple summarization, e.g. what why the Army uses mission command.
- The Falcon model seems to do better at answering complex questions that require synthesis between concepts, for example that the relationship between mission command and fire & maneuver is about flexibility and agility. Whereas the GPT-3.5 model mostly summarizes fire & maneuver (it does make a tentative connection between mission command and flexible coordination, but misses the agility aspect).
- The Falcon model might benefit from a higher repetition penalty.
- The Falcon model was simultaneously domain-tuned and instruction trained, as the examples for training were all derived from the target domain.
- Comparison evaluations seem challenging. Semantic similarity metrics would likely work for summary-type answers. The more intellectually complex answers that require synthesizing concepts might need human or separate eval-LLM to asses.
| Question | GPT-3.5 | Flacon-7b FT | Context |
|---|---|---|---|
| why does the army use mission command? | The army uses mission command as its approach to command and control because it empowers subordinate decision making and decentralized execution appropriate to the situation, supports the Army’s operational concept of unified land operations, and emphasizes seizing, retaining, and exploiting the initiative. Mission command is based on the Army’s view that war is inherently chaotic and uncertain, and that subordinate leaders often have a better understanding and are better poised to make decisions at their level during a battle. Successful mission command is enabled by the principles of competence, mutual trust, shared understanding, commander’s intent, mission orders, disciplined initiative, and risk acceptance. | The Army uses mission command as its approach to command and control that empowers subordinate decision making and decentralized execution appropriate to the situation. Mission command supports the Army’s operational concept of unified land operations and its emphasis on seizing, retaining, and exploiting the initiative. The mission command approach to command and control is based on the Army’s view that war is inherently chaotic and uncertain. No plan can account for every possibility, and most plans must change rapidly during execution to account for changes in the situation. No single person is ever sufficiently informed to make every important decision, nor can a single person keep up with the number of decisions that need to be made during combat. Subordinate leaders often have a better understanding and are better poised to make decisions at their level during a battle. Successful mission command is enabled by the principles of competence, mutual trust, shared understanding, commander’s intent, disciplined initiative, risk acceptance, and disengagement criteria. | COMMANDER AND STAFF ORGANIZATION AND OPERATIONS, (2022, May 16). FM 6-0, https://armypubs.army.mil/epubs/DR\_pubs/DR\_a/ARN35404-FM\_6-0-000-WEB-1.pdf MISSION COMMAND: COMMAND AND CONTROL OF ARMY FORCES, (2019, July 31). ADP 6-0, https://armypubs.army.mil/epubs/DR\_pubs/DR\_a/ARN34403-ADP\_6-0-000-WEB-3.pdf |
| How does mission command enhance fire & maneuver? | The provided context information does not provide a direct answer to the question of how mission command enhances fire and maneuver. However, it does mention the importance of coordination between fires and movement in massing combat power, as well as the use of fires to weaken the enemy’s position and destroy key enemy forces. Effective maneuver also requires close coordination of fires and movement. Mission command, as a doctrine that emphasizes decentralized decision-making and the empowerment of subordinates, may contribute to this coordination and effective use of fires and maneuver. | Mission command enhances fire & maneuver by providing commanders with the ability to coordinate and synchronize multiple elements in the field. This allows for a more flexible and adaptable approach to achieving tactical objectives, which is essential in modern warfare. Additionally, mission command enables commanders to take advantage of emerging opportunities and respond quickly to changing situations. By utilizing digital technology, commanders can share information and make decisions faster than ever before. Overall, mission command is a critical component of modern military tactics and strategy, and its benefits are numerous. | BRIGADE COMBAT TEAM, (2021, January 19). FM 3-96, https://armypubs.army.mil/epubs/DR\_pubs/DR\_a/ARN31505-FM\_3-96-000-WEB-1.pdf MANEUVER ENHANCEMENT BRIGADE, (2021, November 9). FM 3-81, https://armypubs.army.mil/epubs/DR\_pubs/DR\_a/ARN34192-FM\_3-81-000-WEB-1.pdf OPERATIONS, (2022, October 1). FM 3-0, https://armypubs.army.mil/epubs/DR\_pubs/DR\_a/ARN36290-FM\_3-0-000-WEB-2.pdf |
| Tell me in detail about commander’s intent. | Commander’s intent is a clear and concise expression of the purpose of the operation and the desired military end state that supports mission command. It provides focus to the staff and helps subordinate and supporting commanders act to achieve the commander’s desired results without further orders, even when the operation does not unfold as planned. The higher echelon commander’s intent provides the basis for unity of effort throughout the force. Each commander’s intent nests within the commander’s intent two levels up. During planning, the initial commander’s intent drives COA (Course of Action) development. During execution, the commander’s intent establishes the limits within which a subordinate may exercise initiative. The commander’s intent succinctly describes what constitutes success for the operation. Commanders convey their intent in a format they determine most suitable to the situation. It may include the operation’s purpose, key tasks, and conditions that define the end state. When describing the purpose of the operation, the commander’s intent does not restate the “why” of the mission statement. Rather, it describes the broader purpose of the unit’s operation in relationship to the higher commander’s intent and concept of operations. Key tasks are those significant activities the force must perform as a whole to achieve the desired end state. Key tasks are not specified tasks for any subordinate unit; however, they may be sources of implied tasks. | The commander’s intent is a clear and concise expression of the purpose of the operation and the desired military end state that supports mission command, provides focus to the staff, and helps subordinate and supporting commanders act to achieve the commander’s desired results without further orders, even when the operation does not unfold as planned. It includes initial key tasks. In addition to the expanded purpose of the operation and the desired end state, it includes initial key tasks. The higher commander’s intent provides the basis for unity of effort throughout the force. Each commander’s intent nests within the higher commander’s intent. The commander’s intent explains the broader purpose of the operation beyond that of the mission statement. This explanation allows subordinate commanders and Soldiers to gain insight into what is expected, what constraints apply, and most importantly, why the mission is being conducted. Based on their situational understanding, commanders summarize their visualization in their initial commander’s intent statement. The initial commander’s intent links the operation’s purpose with conditions that define the desired end state. Commanders may change their intent statement as planning progresses and more information becomes available. A commander’s intent must be easy to remember and clearly understood by leaders two echelons lower in the chain of command. The more concise a commander’s intent, the better it serves these purposes. Typically, a commander’s intent statement is three to five sentences long and contains a purpose, key tasks, and end state and is briefed at the end of a mission analysis. | MISSION COMMAND: COMMAND AND CONTROL OF ARMY FORCES, (2019, July 31). ADP 6-0, https://armypubs.army.mil/epubs/DR\_pubs/DR\_a/ARN34403-ADP\_6-0-000-WEB-3.pdf STAFF REFERENCE GUIDE VOLUME I, UNCLASSIFIED RESOURCES, (2020, December 7). ATP 5-0.2-1, https://armypubs.army.mil/epubs/DR\_pubs/DR\_a/ARN34870-ATP\_5-0.2-1-000-WEB-1.pdf OPERATIONS, (2022, October 1). FM 3-0, https://armypubs.army.mil/epubs/DR\_pubs/DR\_a/ARN36290-FM\_3-0-000-WEB-2.pdf |
| How does commander’s intent inform intelligence operations? | The provided information does not directly answer the question of how commander’s intent informs intelligence operations. However, it does mention the importance of intelligence in decision making, the need to balance risks and opportunities, and the role of intelligence officers in managing priority intelligence requirements for the commander. It also discusses the challenges of information collection in the operational environment, including the need to fight for intelligence against enemy formations and sophisticated threat capabilities. | Commander’s intent informs intelligence operations by providing guidance to subordinate units on what to collect and how to use collected information. It also provides focus and direction to the intelligence effort, ensuring that intelligence supports the commander’s objectives and priorities. For example, if the commander’s intent is to defeat an enemy force, then intelligence should focus on identifying the location and composition of enemy forces. Similarly, if the commander’s intent is to gather information about the enemy’s movements and intentions, then intelligence should focus on tracking enemy troop movements and gathering information about enemy intentions. In addition to providing focus and direction, commander’s intent also ensures that intelligence is timely and accurate. By specifying the desired end state, the commander ensures that intelligence is relevant and actionable. For example, if the commander’s intent is to locate and destroy enemy artillery, then intelligence focused on locating and destroying enemy artillery would be more useful than intelligence focused on other aspects of the enemy’s military capability. Finally, commander’s intent provides guidance on how to integrate intelligence into the overall scheme of operations. By articulating the desired end state, the commander enables subordinates to prioritize intelligence collection and processing activities. For example, if the commander’s intent is to locate and destroy enemy artillery, then intelligence focused on locating and destroying enemy artillery would be prioritized over intelligence focused on other aspects of the enemy’s military capability. In conclusion, commander’s intent plays a critical role in informing intelligence operations by providing focus and direction, ensuring that intelligence is timely and accurate, and enabling integration into the overall scheme of operations. | OFFENSE AND DEFENSE, (2019, July 31). ADP 3-90, https://armypubs.army.mil/epubs/DR\_pubs/DR\_a/ARN34828-ADP\_3-90-000-WEB-1.pdf OPERATIONS, (2022, October 1). FM 3-0, https://armypubs.army.mil/epubs/DR\_pubs/DR\_a/ARN36290-FM\_3-0-000-WEB-2.pdf MILITARY INTELLIGENCE (MI) COMPANY AND PLATOON REFERENCE GUIDE, (2021, March 9). TC 2-19.01, https://armypubs.army.mil/epubs/DR\_pubs/DR\_a/ARN31806-TC\_2-19.01-000-WEB-1.pdf |
You must log in or register to comment.